[Texto original en inglés. En los próximos días incluiré la traducción.]
Summary: A few questions to guide the policy implementation in the digital space
Legislation drafting and rule making
- Does the process of drafting the legislation have the involvement of those tasked with implementing later on?
- Does the legislation focus on setting goals/targets or on establishing procedures?
- How flexible/realistic is the timetable set by the legislation?
The main goal of these questions is to identify whether the underlying process is “waterfall” or “agile”. In many cases, especially in the digital/IT space, there appears to be a disconnection between the policy advocates/designers and the policy implementers. A waterfall approach, in which the policy process is sliced in strictly sequential part aggravate this problem and keeps implementers out of the crafting of legislation. The first question therefore seeks to point out the tendency to engage in a waterfall approach when an agile might be more appropriate.
The second question continues with this overall approach by focusing on goals and targets. A legislation process that does not involve implementers tends to put them in straightjackets, with little room to maneuver. The problem with this approach is that implementers are better qualified to identify the key and necessary features of a project, so if they are not going to be part of the drafting process the legislation should at least give them enough latitude and space to act and make modifications as necessary.
Finally, the third question highlights the issue of time. Does the legislation provide enough time to get things right? From both a technical and political perspective, it is better to deliver late than to fail on time. Policy designers tend to systematically underestimate the time necessary to make a product work; moreover, since most of digital is designed with the goal of maximizing ease of use, this leads those outside this space to take some features for granted. Time is always an issue in implementation, but in a digital project even more so.
- Does the RFP ask for the right thing? Is it considering all possible options available?
- How can we ensure principal-agent problems? How can we motivate good performance after awarding the contract?
The first question focuses on making sure that the RFP is open enough to allow for creative solutions that can improve upon initial expectations. If the RFP is too restrictive, then vendors will submit proposals that only deviate slightly from the specifications, even if there are better solutions available. RFPs that are too restrictive will therefore “miss out” on better proposals.
The second question is thornier. Specifically, during the RFP process vendors have an incentive to put their best effort and try their best to get the contract. Yet once the contract is signed, what ensured that the vendor has the incentives to try its best? It can simply stick to deliver what is stipulated in the contract, even if more can be done.
Implementation and accountability
- Where does the buck stop? Who is empowered to make important decisions about the product’s features? Does it require approval from the very top?
- How can we get early feedback and act on it?
Finally, there are two questions that pertain to the implementation process and how teams are structured. For instance, who is empowered to make key decisions at each stage? We have seen that, in the case of healthcare.gov, relatively simple decisions such as naming features like the exchanges required approval from the president. This can lead not only to major delays in implementation, but also to conformism throughout the team: If things seen as relatively minor take so much effort to be changed, then why bother?
The last question brings us full circle on the issue of waterfall vs. agile implementation. Arguably the main benefit of agile implementation is that it is able to gather feedback rather quickly. It is therefore important to ask how can the team act on it and make changes as necessary. The experience of healthcare.gov shows that, when feedback is not gathered before launch, the rollout can lead to ugly surprises. Therefore, the team needs to be able to act quickly upon new information and adjust the product as necessary.
Analyze: What are the key takeaways?
From my point of view, the main challenge of digital implementation is to make the policy design process match the policy implementation. This is generally the case of any policy, digital or otherwise, but it appears to be more so with digital projects for a few reasons. First of all, digital implementation seems to be taken for granted. Because most people’s experience with digital projects involves already polished products that emphasize ease of use, this gives the inaccurate impression to many that to reach said level of usability is relatively straightforward, when it is not. Second, digital implementation often requires a lot more user feedback as early as possible in the process, something that is not usual in many other instances and thus is not contemplated when designing a policy and planning its rollout. And finally, digital implementation involves some very specific and specialized knowledge that is less common among older generations that are nevertheless tasked in many cases with crafting legislation.
Given this, what are the conditions necessary for successful implementation of an IT project? If the case of healthcare.gov is any indication, it seems that a clear organizational structure, with clear accountability at each level and where team members are not afraid to raise their voice and point out flaws is key. This probably means that each sub-team tasked with a specific function should probably remain small (say, no more than 9 or 10 members), and that leaders at the top foster collaboration. Another key condition, in light of the challenges above, is to communicate constantly with those in charge of policy design in order to bridge the divide that often exists between the two sides. If some stakeholders have a tendency to systematically underestimate either the difficulty or the necessary time for successful digital implementation, then it is necessary to spend some time and resources communicating with them in order to ensure that unreasonable timetables are imposed.
Synthesize: How can we apply this to labor regulation in Mexico?
I have never worked in the digital space, but while working on my Second Year Policy Analysis (SYPA) I encountered a specific instance in which the lessons of digital implementation became very relevant. I worked on a mechanism to reduce labor informality in Mexico, and as part of the policy I proposed a program that included a workshop as well as a series of internships for non-college educated workers. The program, run by the government, would provide these workers with the opportunities and networking necessary to get a formal job. The question, however, was how to get said workshop going, and a digital format was one of our options.
In order to save some time and resources, the idea was that the admission to the program, the workshop and the final examination would take place online. To be clear, under this policy design the workshop itself was not as important as the admission and the final examination, which provided a certification to the workers and signaled their abilities to potential employers. Thus, saving resources on the workshop via a website where participants could register sounded like a good idea.
There were some important complications, however. For one thing, if the program was targeted to non-college educated workers, when they a person enrolled online in the program the system would have to screen his or her information to verify that this individual belonged to the target population. In addition, because enrolling in the program required a small payment, it would need to include an option allowing the sure to make it online using a bank account or introducing a code given at the bank after making the payment in person. Finally, because a lot of Internet connectivity in a country like Mexico takes place via cheap smartphones and mobile devices, this meant that the web page required a good mobile version.
Suffice to say, when first formulating this requirements, it sounded easy. I was more concerned with the scale up: Using the framework of Andrews, Pritchett and Woolcock (2014), the plan was to start small involving a few sectors of the economy and a few locations, ad to gradually expand both in the number of participants as well as the scale of the activities to be implemented. However, little by little the challenges of effectively formulating a webpage that served non-college educated workers began to capture all of my attention. In particular, I wondered how to introduce an agile framework into the design of the webpage required for the workshop.
How can I do this in the future as I explore this policy proposal beyond my SYPA? A first step seems to be embed the agile implementation within the overall framework on scalability of Andrews, Pritchett and Woolcock (2014). The idea is that the earlier, low-scale stages of the policy, both at the quantitative (number of users) as well as functional (number of activities) serve as platforms for trying different designs and getting early feedback. Moreover, because many of the workshops and internship programs will be strictly local, different alternative designs could be built by two or three different vendors in order to assess which one delivers the best user experience–after all, even if the workshop is not as important from a policy perspective, program participants should find it easy to participate in them online. By blending the scaling up of the program with an agile approach, we could learn more about user behavior and tailor the program itself better.
A second step is to create controls in order to avoid what in PDIA is known as “premature load-bearing”: Imposing unreasonable/unrealistic demands that only end up contributing to kill the impetus and energy to achieve results. This harkens back to the problem with timetables in IT that become straightjackets–this is the digital version of premature load-bearing. As the website to set up the workshop is created, it is important to administrate expectations about what the webpage will able to do in its early stages: Maybe a first step should only focus on successful registration into the program, while keeping the workshop to in-person sessions. Afterward, as the work on the website consolidates, the program could move gradually to put some material on the webpage, or to complete assignments on the web, until the full workshop is moved to the web. This gradual transition fits in nicely with the paradigm of iteration that is at the heart of PDIA.