Why career transition is so hard. And how to manage it better.

Why change is hard

Some of what makes changing careers difficult will be unique to you and your particular circumstances. But you’ll almost surely have to confront two challenges while in transition: a lack of institutional support and an unsettling loss of professional identity.

Lack of institutional support

Until recently, many of the important moves in our working lives were institutionalized, meaning they were well scripted by the communities and professions that oversaw them. If you wanted to become a doctor, make partner at a law firm, or move up the management ranks, you had to follow a clear sequence of steps. From schooling through retirement, you knew more or less how long each step would take. Peers went through them with you, and elders showed you the way. Throughout the process, gatekeepers marked your progress with degrees, credentials, promotions, and, eventually, gold watches.

Today, with the rise of non-linear career paths, many of the transitions we make are ‘under-institutionalized’. There is no immutable series of steps for the change you need to make, and no telling how long it will take or how to measure your progress. Complicating matters, the direction of travel is often from large organizations, which have well-structured recruiting and hiring processes, to smaller players, private firms, and entrepreneurial opportunities in unstructured job markets. Increasingly, too, people are shifting from full-time jobs to fluid, individualized portfolios of gigs and part-time roles. All this can offer you an endless sense of possibility, but it also makes it harder to figure out what you want to do, and where to start.

Loss of professional identity

Decades’ worth of research in social psychology shows that our sense of identity is anchored in the well-defined groups and organizations with which we’re associated and by which we are recognized. Without the cover and support of a traditional employer and a stable work identity, we can quickly start to feel lost, anxious, irrelevant, and insecure. Those feelings can be strongly amplified if instead of choosing to leave your job, you’ve been let go or fired.

Emotional ups and down are a standard feature of any transition. Unfortunately, you’re likely to have to cope with them longer than you might expect, for several reasons. If you’re an experienced executive, you probably have some very specific skills and knowledge, and that can make finding a new fit challenging, especially because many senior positions are not advertised publicly. Vetting and interviewing processes have also gotten longer — the result of a confluence of factors, including the use of multiple screening methods such as personality tests and skills assessments. More broadly, the current economic uncertainty can make it slower and harder to secure a job or financing for an entrepreneurial venture.

Search different

When it comes to making big life decisions, we’re typically told that the first step is to figure out what it is that we really want. Once the hard work of self-reflection is done, the thinking goes, the rest is simply a matter of implementation.

The problem is, that approach doesn’t work. Year in and year out, I’ve seen the same dynamic: people know what they don’t want to do anymore, or what is no longer viable, but they don’t know what to do instead. So they delay getting started, feeling that they first need greater clarity or waiting until they’ve lost a job and are forced to make a change.

It’s easy to blame this inertia on human psychology: we fear change, lack maturity, don’t understand ourselves, have failed to reflect on our true nature. But my research has led me to a different conclusion. Usually we fail to change simply because we don’t know how to go about it. The problem lies in our methods, not our minds.

Career change is iterative. You can’t line everything up in advance. You have to figure things out over time and make adjustments as you go.

The story of Thomas (a pseudonym) is instructive. Thomas had a conventional career in finance with a Fortune 500 company until he reached the age of 48. Each year he got bigger bonuses, greater performance evaluations, and bigger responsibilities, and he was eventually promoted to finance director in his company’s healthcare business. But when that business was dismantled, he lost his job, and finding a new one proved difficult. He couldn’t make a lateral move, because roles at his level were few and far between; he didn’t yet have the experience to become a CEO; and he was told he was overqualified and ‘too well paid’ for more junior roles. Eventually he got an offer from a mid-size firm that needed his expertise temporarily. It wasn’t an ideal match, but he took the position to avoid having a big gap on his CV.

After that gig ended, Thomas was back to square one. But during the search process he’d reconnected with a couple of old friends and colleagues in similar straits who had pooled office space and launched freelance advisory businesses that allowed them to generate some income while searching for the next thing. Thomas collaborated with them on two start-up ideas, but neither really took off, so he and his family made some adjustments: his wife ramped up her career, and they moved to a less costly city and invested in profit-generating rental properties.

Over those difficult years, Thomas began thinking about a market gap he had noticed while working for his previous employer: the dearth of assisted-living facilities for the ageing population in a region he knew well. Gradually he made connections with the players in that space, looked at models in other countries, and began helping companies orchestrate deals in the sector. Over time he became a go-to person for this kind of work. Now, with interest rates rising and the deal flow slowing, he’s seeking an operational role again, keen to use the skills he’s acquired.

What can we learn from Thomas’ story? After he was forced into a career change, he didn’t begin by trying to figure out who he ‘really’ was and map out a plan before moving forward. Instead he hustled, following his nose and activating his networks, trying lots of different things, often simultaneously, without fully settling on one. He learned and adapted, which is what we all have to do in a world that rewards optionality and the exploration of many possible selves.

A liminal state

Embracing optionality and multiplicity in the search for a new career, as Thomas did, makes a lot of sense intellectually, but emotionally it’s a rollercoaster ride. That’s because it puts you in what anthropologists call a ‘liminal state’, where you must navigate between a past that’s clearly over and a future that’s still uncertain.

Liminality can be unpleasant, especially for those of us used to single-mindedly pursuing clear goals on a well-trodden path. But when you’re changing careers, liminality gives you the necessary time and space to question the old givens. Think of it as an identity time-out, when you can let go of your commitment to who you used to be and focus more creatively on who you might become. It takes time to discover what you want to change, identify the habits and assumptions that might be holding you back, and build sufficient skills experience and connections in a new arena. So instead of trying to land your next role as quickly as possible, you must embrace liminality. You have to be willing to get and stay lost for a while.

My research has shown that there are three important ways to make the process easier.

Diverge and delay

Finding your next role almost always takes longer than you expect. Make the most of that time. A traditional plan-and-implement program is likely only to get you more of the same. If you want your liminal period to lead to real discovery, you need to experiment with divergent possibilities while delaying commitment to any one of them. In doing so, you’ll have to think more creatively and will obtain more information about yourself and your options.

Consider the case of a lawyer I’ll call Sophie. Coming out of a two-decade corporate career, Sophie wasn’t sure what she wanted to do next, but she was keen to explore a range of possibilities, among them documentary filmmaking, non-executive board roles, and sustainability consulting. Over a three-year period she got herself a board-director accreditation, took filmmaking and journalism courses, worked on a start-up idea, did freelance consulting on compliance, joined a non-profit board, did an internship in a newsroom, completed a corporate film project in her old area of ethics and integrity, and produced (and won prizes for) two short films. With deeper insight about the economic realities of the media and entertainment industries, she now feels well placed to decide between putting most of her energy into documentary filmmaking and building a more diverse portfolio of work.

Exploit and explore

Human beings are very good at either-or thinking: either I’m leveraging my old skill set or I’m pivoting to something new. But most people making a career transition have to do both simultaneously, at least at first — ideally staying in their old jobs and careers while exploiting and exploring on the side until something new becomes viable.

One investment banker did just that when he began to tire of his career. While still working at his blue-chip firm, he started a blog lampooning the industry under the pen name Litquidity, shortened to Lit. Next, after switching to private equity, he launched the Litquidity Instagram account. As his following grew, he thought about turning his side gig into a full-time one but didn’t feel he could take the risk.

Reluctantly, he returned to banking. But just a few years later the Litquidity business had grown enough for him to make that transition. Today Lit is a social media sensation, with several Instagram accounts, a popular newsletter, a podcast, a deal with a recruiting firm, and roles as an angel investor and venture capital scout for Bain Capital Ventures.

Bridge and bond

We grow professionally in and through our relationships with others. But when it comes to career change, the connections we already have aren’t usually all that help- ful. You need to build your relationships in two ways: by bridging, which involves creating or reactivating relationships beyond your current social circle, and bonding, which involves deepening ties and finding community within a close circle of kindred spirits.

Bridging often gets more attention because the golden rule of job-search networking has always been to mobilize your weak ties — the relationships you have with people you don’t know well or don’t see very often — to maximize your chances of learning about new opportunities. A recent study of more than 20 million LinkedIn users showed that this is indeed true: weak ties help you get a job because they connect you to farther-flung social circles. The hitch is that most people dread the prospect of reaching out to their extended network in this way because it’s hard work that can leave you feeling exposed and vulnerable.

That’s why bonding relationships — critical for anyone trying to stay healthy, happy, and sane — are so important for people in transition. They provide the support, sustenance, and space people need to process the unsettling emotions of the transition period.

There’s no one way to forge bonding relationships. Sometimes the important ones are with people you already know (especially spouses), but sometimes they are with kindred spirits, people also in transition, or those already working in the field to which you aspire. Thomas bonded with the buddies he shared office space with. Sophie bonded with the newsroom team (who invited her to stay on a freelance basis after her internship), a community of female documentary filmmakers, and some of the other mature students in her courses. There’s convincing evidence that when independent consultants find their tribe, they are not only more productive but also better able to tolerate the anxiety of being on their own. One way or another, we all need a secure base from which to explore the unknown and turn painful feelings into sources of creativity and growth.

The learning plot

Being in transition is like losing the plot of your professional life. You need to diverge and delay, exploit and explore, and bridge and bond to find a new narrative thread. In doing so it’s essential to engage with others and tell them your story — again and again, as much to make sense of your experience as to enlist their help. This process, which I call ‘self-reflecting out loud’, will propel you forward, in no small measure because others will respond, sympathize, commiserate, ask questions, call your bluffs, and share their own experiences in ways that will help clarify your thinking.

Your story will most likely depart from the timeless myth in which a hero (you!) struggles to pivot to the next career and, by dint of hard work and determination, ultimately finds a happy ending. That kind of simple, linear plotline doesn’t reflect the realities of today’s working world, in which jobs and careers are precarious, liminality can be long, and resolution — if there is any — tends to be short-lived. So we all need to get comfortable with a new kind of narrative that revolves around what I call ‘the learning plot’ — a story of ongoing struggle and adaptation. That’s what Thomas, Sophie, and Lit all did.

As constant reinvention becomes the norm, the stories that define us have no start or ending. Instead of closure, the prize is learning: what we learn about ourselves when we embrace, rather than resist, the loss of status and identity will give us access to more options in the long term. Proficiency in being liminal won’t reduce the great uncertainty before you. But it will increase your capacity to successfully navigate the present and future transitions that are the signature of a modern career.