Learning from mistakes—your own and those made by others—is a mark of effective organizations. Fostering a learning environment means seeing every possible outcome as a learning opportunity. And every project manager knows the importance of documenting and continuously reviewing these lessons learned.
Many companies maintain databases for their lessons learned which contain heaps of valuable information, potentially critical to the success of new projects. More often than not, though, they are incredibly difficult to search and navigate, rendering them unusable.
Even mature and knowledge-driven organizations like NASA find themselves struggling to find the answers they need in their own knowledge bases. Nothing illustrates that better than the infamous case of the “lost tapes”, as they came to be referred to by the press.
In the early 2000s, a team of retired NASA employees set out on a search after the tapes containing the original footage of the Apollo 11 moonwalk by Neil Armstrong and Buzz Aldrin. After an exhaustive, three-year search, NASA was forced to admit to an embarrassing mishap: the tapes were most likely improperly labeled and ended up being erased and reused at some point in the 1980s.
While this case is certainly an outlier, it does reflect the challenges that come with managing knowledge. NASA famously maintains an automated database called the Lessons Learned Information System (LLIS). It contains impressive volumes of data collected from past tests and missions, both successful and failed, and is used in the planning of future projects and expeditions into space. As the database absorbed more and more information, it became apparent to NASA that in order to maintain its usability the system needed to be modernized.
Collecting and storing the lessons learned is only half the battle. Making that knowledge easily discoverable is the real challenge. David Meza, NASA’s Chief Knowledge Architect understood this all too well as he struggled to find answers in LLIS. The system required you to punch in a keyword which would then produce an endless, randomly arranged list of links to documents, every one of which needed to be checked one by one—a process so tedious that NASA engineers hardly ever consulted the system.
Meza saw the tremendous potential in the database and embarked on a mission to unlock it. He began by experimenting with networks and graph databases. Each individual lesson learned became a node in the network, clustered around a relevant topic using a machine learning algorithm. That data was then stored and organized using the database system Neo4j, and a graph visualization was used to visualize the connections between the clusters. This approach allowed users to explore the data in a new, intuitive way, reducing the time they needed to find answers.
The new approach immediately yielded positive results. In 2014, when a team of NASA engineers was working on Exploration Flight Test-1, an unmanned mission to send the Orion spacecraft into orbit, they discovered that Orion’s uprighting system wasn’t working correctly. Knowing that Apollo used a similar uprighting system to Orion, the team turned to the NASA lessons learned database. An 8-day search turned up only 3 relevant documents, none of which helped them solve the problem. In desperation, the team reportedly even visited retired Apollo astronauts at their homes to see if they had any useful documents stashed away in their attics.
Meza and his team came to the rescue, discovering over 30 relevant files in the database, which ultimately helped NASA find a solution to Orion’s uprighting system problems. Without that information, the team would have had to spend years and millions of dollars testing different designs, delaying Orion’s 2023 launch even further.
Nuclino follows NASA’s approach to knowledge management. A lessons learned database is only useful when the information it contains can be easily found and accessed, making an intuitive, visual interface a critical component of the system. The graph view in Nuclino organizes your knowledge in a mind map with topic clusters, giving you a visual overview of the whole database and making it easy to identify information relevant to your topic or project.
Even if your lessons learned database doesn’t span decades it can easily swell and drown you in information. Experience shows that the deciding factor between the success and failure of a project is not the volume of knowledge at your disposal, but rather its discoverability. When done right, a lessons learned database can tremendously reduce your time to find answers and allow you to start your project on the right foot.
I was in the ballroom of the Mandarin Oriental in Washington, D.C., when the worst pandemic since the 1918 Spanish flu broke out. On cable news, there were reports of four hundred confirmed cases, mostly clustered in Frankfurt, Germany, but with infected individuals reported as far afield as Tokyo, Kabul, and Caracas. Brow furrowed, eyes widened, the anchorwoman’s tone was urgent as she described the spread of a new type of parainfluenza virus, called Clade X. Transmitted through inhalation, it left the infected contagious but otherwise unaffected for up to week before killing more than ten per cent of its victims.
In the ballroom with me, seated around a U-shaped table under glittering chandeliers, were ten senior political figures, an ad-hoc working group convened at the President’s request. The situation looked bad. At Ramstein Air Base, in southwest Germany, three U.S. service members were critically ill, and three infected Venezuelans warned that the outbreak there was much worse than authorities were admitting. According to the Centers for Disease Control and Prevention, a vaccine would likely take more than a year to develop. Meanwhile, Australia, China, and South Africa had already imposed travel restrictions on flights from Germany and Venezuela. A bipartisan group of senators was calling for a similar travel ban; a recent poll had suggested that sixty-five per cent of the public supported them. “What should our priorities be?” the national-security adviser asked.
The room was divided. The health experts at the table warned against closing the borders. “This is an opportunity to do something that will achieve nothing except for diverting resources from real problems,” the Secretary of Homeland Security said. The director of the C.D.C. noted that disrupting global supply chains was a terrible idea, given that most personal protective equipment is imported from overseas. The Secretary of Defense was not convinced. “Look, I’m not talking about going out with our hands waving and our hair on fire,” he said. “But if the President is seen as not taking vigorous action, they’re going to start firing from the Hill.”
Fortunately—and, as you might have guessed by now—this was a fictional outbreak. “Clade X” was a daylong pandemic simulation held by the Johns Hopkins Center for Health Security, in May. The C.D.C. director was being played by a former holder of that job, the Secretary of Defense was the former Republican senator Jim Talent, and the Secretary of Homeland Security was played by Tara O’Toole, a former Under-Secretary in that department. Clade X turned out to be an engineered bioweapon, combining the virulence of Nipah virus with parainfluenza’s ease of transmission. It had been intentionally released by A Brighter Dawn, a fictitious group modelled on the Japanese doomsday cult Aum Shinrikyo, which carried out the sarin-gas attacks in the Tokyo subway system, in 1995. A Brighter Dawn’s stated goal was to reduce the world’s population to pre-industrial levels. By the end of the day, which represented twenty months in the simulation, they had managed to kill a perfectly respectable hundred and fifty million people. “America was just wiped out,” Talent said, before heading to a post-event cocktail reception.
This is the third major pandemic exercise that the Center for Health Security has run. The first, called Dark Winter, was held in 2001 and simulated a smallpox attack on Oklahoma. Its timing, just a few months before 9/11, made its terrifying outcome—the near-complete breakdown of government and civil society—deeply resonant. Dark Winter is credited, in part, with spurring George W. Bush to pass Directive 51, a largely classified plan to insure the continuity of government in the event of a “catastrophic emergency.”
“I would say there has been enormous progress in our preparedness since then,” Tom Inglesby, the center’s director, who played the national-security adviser in Clade X, told me. These advances include new preparedness programs and offices at the C.D.C. and the Department of Health and Human Services; national stockpiling of vaccines and medications; and, at the international level, investments in emergency financing and infectious-disease infrastructure. According to an article that appeared last year in the British Medical Journal, however, the world remains “grossly underprepared.” Philanthropist-in-chief Bill Gates drew on models developed by the Institute for Disease Modeling, a venture founded by his former Microsoft colleague Nathan Myhrvold, to warn that, at our current state of readiness, roughly thirty-three million people would die within the first six months of a global pandemic similar to the 1918 flu.
“Even with the progress that’s been made, there are a lot of serious vulnerabilities,” Inglesby told me. “The reason we wanted to do this now is because there are new leaders, and there are still large issues that have not been resolved.” Some of these issues emerged during the Ebola crisis of 2014, when the United States came very close to shutting down air travel from the West African countries where the disease had broken out. Such a ban would have been in violation of the World Health Organization’s International Health Regulations, to which the U.S. is a signatory; it also, in the opinion of most health professionals, would be both ineffective and counterproductive. “To contain a nasty outbreak like that, you need to move people; you need to move assets; you need labs, equipment, supplies, medicine, food all coming into West Africa,” Inglesby explained.
But, while scientific opinion on the wisdom of travel bans is established, political and popular instinct lags behind—and it was here that the value of Clade X became apparent. As the experts at the table discussed the issue, it was clear that even those who initially leaned toward a ban, such as Talent, were perfectly willing to follow expert advice; they were just extremely sensitive to the pressures of public perception. Telling the average American that a travel ban wouldn’t work “is not going to wash,” Talent said. “This needs to be communicated in terms of their health—that a travel ban will detract from our ability to protect their health.” None of the health professionals at the table had thought to frame their opposition to the ban this way, which left the politicians without the rhetoric they needed to reassure their constituents.
Many of the day’s surprises involved the triumph of these kinds of intangible considerations over rational decision-making. Presented with a scenario in which limited supplies of the vaccine were going to be given to essential workers, including members of Congress and business leaders, rather than children and pregnant women, the table revolted. As acting Attorney General Jamie Gorelick put it, the correct advice may well be to put your own facemask on first before helping others—but leaving babies unprotected is not an option. Meanwhile, faced with a request for military support from Jordan, whose government, weakened by the outbreak, was under siege from terrorist groups, the group was unanimous in agreeing that the United States must continue to support its allies in the Middle East, despite the medical risk and the fact that eighty per cent of Americans thought that the mission was a bad idea. “A number of the players, a number of times, said the equivalent of ‘We can’t do this; it’s not consistent with American values,’ ” Inglesby said. “When you’re having a public-health debate about quarantine, say, or resource allocation, you don’t always go to that level.”
Some of the day’s dilemmas revealed vulnerabilities that are hardwired into the American system. Some private hospitals, for instance, turned away Clade X patients in order to protect their shareholders. (By the end of the simulation, American health care had been forcibly nationalized.) Governors enacted state-level quarantines and border cordons. “In a serious outbreak, there will be federalism issues,” Inglesby said. “They may be manageable. They may not be.”
Several challenges, however, were ones that could be solved with more planning and investment. Julie Gerberding, a former director of the C.D.C., pointed out that global vaccine-manufacturing capacity is insufficient to meet projected demand during a worldwide pandemic. Margaret Hamburg, a former commissioner of the Food and Drug Administration who was playing the Health Secretary, argued in favor of an idea that has been floating around without funding for years: a clinical corps, equipped to travel to the source of outbreaks and try to contain them there before they reach pandemic proportions.
Even within the artificial confines of the simulation, there was a lack of leadership. Everyone agreed that the President had the final word on everything in general, but nobody seemed to be responsible for America’s outbreak response specifically. The table returned endlessly to questions about who would brief Congress; who would be capable of authorizing an emergency deployment of military tents as civilian isolation units; who would call state governors to try to ensure a coördinated national response; and who, even, would attend all the funerals. As the former senator Tom Daschle, playing the Senate Majority Leader, complained, halfway through the day’s exercise, “We’re five months into this crisis and I still can’t tell you who’s in charge.” Ironically, just a couple of days earlier, the person to whom this responsibility would have fallen in real life, Rear Admiral Tim Ziemer, had been removed from his position as the head of global health security on the White House’s National Security Council after the Trump Administration cut funds for fighting pandemics.
“In a way, it’s sad that we’re still having these conversations,” Gerberding told me, during a pause in the proceedings. Back in 2001, she helped lead the C.D.C.’s response to the anthrax attacks of that year. “We have a Department of Defense, we fund it pretty well, and it’s pretty stellar,” she said. “We still don’t have that for this kind of defense.”