Policy Prototypes: How designers and policy practitioners can use prototypes to get feedback and iterate on policy

By Angelica Quicksey, Head of Public Spaces Incubator, New_ Public & Chris Meierling, Director of Experience Design, Medable

As designers, we cheer every time we see a government agency embrace the principles and practices of human-centered design. Humancentered design incorporates feedback from the people you’re designing for throughout the design process so you can build unified, clear, and respectful products and services. When making policy decisions for laws, regulations, or program operations, human-centered design practices like prototyping can help you test your assumptions early on to create better outcomes. In our experience, prototyping is an underexplored and under-utilized approach in the public sector. While building robust feedback loops into policy and program development practices will develop stronger policies, government agencies tend to be risk-averse and rely on ideas that maintain the status quo. Prototyping pushes them to be more expansive and test ideas that might not work. But prototyping actually reduces risk. It reveals new opportunities for improvements and success that can’t be surfaced by doing things the way they’ve always been done.

Narrowing the gap

Prototypes are most commonly used while building software. Atlassian—a company that makes collaborative software for agile development teams—defines a prototype as “an early sample, model, or release of a product built to test a concept or process or to act as a thing to be replicated or learned from.” In an agile development process, a prototype is a working piece of code that developers deploy in a matter of days or weeks. This allows developers to get feedback fast.

In the policy world, the feedback loop is much slower. Years may pass between when a policy is first created and when the public and civil servants experience the impacts of that policy. Because policy decisions are made based on the available data—data that can be years old or incomplete—assumptions are unavoidably baked into the development of any policy. The world can change significantly between when an idea is conceived and when it is deployed. The assumptions that get cemented into a program early in the process may no longer be relevant. Prototyping allows us to get feedback more quickly, so assumptions can be tested and policy decisions can be made based on up-to-date, relevant data about what people actually need.

Applying prototyping to policy

We define a prototype as an object, device, or experience that proves or disproves an assumption. A prototype should involve setting up a team to learn and iterate on their policy or the execution of that policy. Policy prototypes are tangible artifacts or experiences that make a policy real for people and allow policy-makers to test their assumptions and iterate based on what they learn. Policies create the frameworks and rules for government programs. People don’t experience a policy until it is delivered through a program, process, or touchpoint. From Medicaid health insurance (a program), to applying for affordable housing (a process), to a letter from a student loan provider (a touchpoint), policies are just words on a page until they’re delivered to the people they are meant to serve. There are three types of prototypes we’ll be discussing:

1. Program prototypes: A government program is a large set of related activities with a particular long-term aim, supported by a local, state, or federal government. Program prototypes let you test and iterate on core program components. This helps you refine and adapt program models toward their intended outcomes, rather than simply the approval or discontinuation of a program.

2. Process prototypes: A process is a sequence of steps to accomplish a task, like the process to register to vote or apply for a benefit. Often processes are part of larger programs and can be represented as a “customer journey.” Process prototypes test the steps in sequence and consider all the parties involved. They can reduce pain points, drive greater efficiency, or increase access.

3. Touchpoint prototypes: Each interaction with a government service or office can be considered a touchpoint, from a single email to a conversation with the agent at your local Department of Motor Vehicles (DMV). Touchpoint prototypes are designed to test and improve specific artifacts or moments of interaction through which people engage with your policy, program, or process.

Policy-makers can create prototypes of programs, processes, and touchpoints before they’re embedded in our everyday lives.

Testing and improving

Many policies result in programs—a large set of related activities with a particular long-term aim, supported by a local, state, or federal government. Examples of programs range from federal workforce development programs from the Workforce Innovation and Opportunity Act (WIOA) policy that distribute resources to states and counties, to city-based housing access programs like San Francisco’s affordable housing program, DAHLIA. A program prototype allows public servants, government officials, philanthropic funders, and the people directly impacted by programs to iterate—to test and improve—on these large, complex initiatives. Iterating helps build better services and inform policy-making upstream. Policymakers are likely familiar with the idea of launching a “pilot program,” a scaled back version of a larger program that either succeeds or fails. (Though they are often expected to succeed.) By contrast, prototypes are even smaller experiments, on limited feature sets, with faster feedback loops, and opportunities for improvement. Failure of a prototype does not mean program failure; it’s merely an opportunity to adapt the prototype (and program) based on the feedback gained, and try again. Given the scale, complexity, and impact of programs, program prototypes are often a collection of many, smaller prototypes: a coordinated effort of small, quick tests and iterations that support larger decisions about what a program or policy should look like. Program prototypes minimize risk by identifying potential failures or hazards early on, so the decision-makers can make adaptations before spending vast resources.

Getting feedback

From procurement and contracting processes to applying for public benefits, civil servants and the public alike engage with the government’s most important functions through processes. A process is a sequence of steps taken to accomplish a task, like the process to apply for public benefits or the process to write and issue a request for proposal (RFP). Often, processes are part of larger programs. Process prototypes can help designers and policy-makers isolate and refine the processes that crosscut their work so that they more effectively and consistently deliver on their intended outcomes. A process prototype simulates the sequence of procedures and transactions needed to fulfill a policy goal. Process prototypes test and refine all the steps in sequence with the actual people involved. By refining processes with a process prototype, civil servants and designers can improve the experience for all the people who use the process, create better program outcomes, and increase trust with the institutions that shepherd those processes.

Testing a process prototype can come through simulations, dry-runs, and walk-throughs to play out unexpected scenarios. (Government folks may be familiar with the term “table-top exercise.”) Just as software development teams dedicate days to looking for places where code will break, administrators should dedicate time to identify overlooked process points.

Testing specific artifacts or moments

When people interact with your policy, they do so through a collection of interactions or artifacts. Each interaction or artifact is a touchpoint. A touchpoint prototype can be as simple as the language in a letter or a new application form. It’s an opportunity to make changes at these moments, test the results of those changes, and make an improvement. For example, many cities have adopted Vision Zero as a policy goal, aiming to reduce traffic fatalities and severe injuries to zero. Cities might create new street signs, introduce fines or new driving rules, or promote changes to large vehicles to make them safer. In this example, a sign is a touchpoint. The moment when a speeding driver pays a hefty fine is also a touchpoint. Both present opportunities for prototyping.

Touchpoints have even smaller scopes than program or process prototypes. Narrow in on one specific artifact or moment that you want to test and what you want to learn from it. Your touchpoint prototype should be narrow enough that you don’t have to iterate on a whole process or program.

But because your touchpoint is likely part of a larger program or process, think about it in that context. What you learn may influence other parts of the program. For example, if your new street sign doesn’t reduce injuries, you may need to try different placements or different signs altogether. So, plan to share your learnings widely early and connect with others who might benefit from them.

Best practices

Understanding best practices and emerging opportunities can help you determine when to use a prototype, who to include in the process, and how to make plans that lead to improved programs, processes, and touchpoints. Avoiding a few common pitfalls will help you roll out initiatives with respect for the public and partners and get the right kind of feedback so your efforts are more successful. As you begin prototyping to test your ideas and assumptions, also consider emerging opportunities to go beyond digital.

Applying best practices

• Understand the policy and organizational landscape before moving forward with a prototype. There may be larger or contributing issues that you aren’t aware of yet. Speaking with people throughout the agency you’re working with will reveal pain points to address. In the case of process prototypes, you can conduct a process audit across your agency to identify the business processes that have the greatest impact on the people who use or rely on them. An audit will help you identify which processes are in most need of change. 
• Partner and align internally to execute prototypes. Even after you understand the landscape, it is challenging to advance change without allies and champions who are directly involved with implementing programs. These very individuals will create the critical mass for implementing new programs, processes, or touchpoints later on. 
• Identify your learning goals early. Prototyping is about learning. If you created a new artifact, but you didn’t learn anything from it, it wasn’t a prototype. Identify the specific qualitative or quantitative indicators that your prototype is meant to impact. In some cases, your learning goal may simply be to understand if you’re moving in the right direction—ie. if your artifact is the right type of artifact at all. Get a baseline measurement of those indicators so you can compare the before and after. You can also co-create prototype indicators with stakeholders to ensure you gather data that matters to policymakers and public administrators and is tied to program outcomes. 
• Partner with a policy expert. The policy landscape around any particular issue can change rapidly and impact how policy recommendations are perceived. Working with a policy analyst or researcher can provide valuable ongoing context for framing the recommendations that result from a prototype. If you move toward implementation, a policy expert can help connect with the champions or critics of a particular type of program or process to unblock development. 
• Implement changes with a small group first and expand incrementally. Starting with a smaller group means that you initially impact fewer people and can make changes more quickly. At each stage in the roll out, you can continue to collect feedback and iterate on the process.

Avoiding common pitfalls

• Your prototype should have a potential path to permanence. A prototype is a promise; if you’re making something to be thrown away, you’re breaking that promise. Your users implicitly trust that you are using their time well and that their input matters. So, though you may abandon a prototype, you shouldn’t test an idea with users that has no potential to become permanent—especially if you are testing an artifact that impacts a vulnerable community. While new barriers can always arise in government, an ethical and effective approach to prototyping considers known barriers in advance.
• Prototypes don’t have to be perfect to get feedback. It’s easy to feel like you need the most polished, high-fidelity version of something in order to get feedback. Though it may seem counterintuitive, this is often not the case. Especially for early feedback, it can be better to test a rough version of an artifact, process, or program design. This is not only because it’s cheaper, but also because people are more willing to give feedback about something that is clearly not finished. 
• Consider potential causes of failure in advance. There are many barriers a policy might encounter: a program design could have already been considered, the population served by the prototype isn’t representative, or certain circumstances might not be replicable in other contexts. Policymakers can forecast potential pitfalls beforehand by imagining the program has already failed and then thinking through what might have caused it to fail. This strategy is called running a pre-mortem. It can be a helpful step for programs that hope to influence policy because you can address the pitfalls your team identifies before implementation begins. 
• Be aware of shifting policy priorities and budgets. Interest in specific programs or policy priorities can rise and fall alongside the legislative cycle or changes in government administrations. Additionally, funds and attention can shift based on changing philanthropic priorities or interests from newly appointed public officials. Consider the timing of your recommendations and think long term about when they might be most influential, as new, and perhaps more relevant, policy advocacy opportunities open in future cycles. Anticipating events like elections, setting target dates to demonstrate outcomes, and building cross-organizational coalitions are all ways to mitigate dwindling support. 
• Focus on outcomes, not outputs. Prototypes might be measured against outputs like number of calls answered or applications reviewed. But these outputs don’t necessarily measure the quality of service provided to a user. The number of customer problems resolved or the rate of error-free applications may be more valuable measures. Understand the difference so you can focus on improving prototype outcomes and, ultimately, program outcomes.

Keep an eye on emerging opportunities

• Prototyping isn’t all about digital. It’s valuable (and increasingly common) to test non-digital programs, processes, or touchpoints. These can include paper communications, like fliers and letters about health insurance enrollment, or in-person interactions, like when a couple applies for a marriage license at City Hall. 
• Use demonstration sites to imagine a future where a policy has already been implemented. Demonstration sites are local sites that serve as a “laboratory” to identify and solve problems that may arise during program implementation. San Francisco’s Safe Injection Demonstration Site sought to do just this. Like paper prototypes, the demonstration site was designed to quickly build empathy for the public’s experience within community and policy stakeholder groups. When paired with campaigns and long-term advocacy, demonstration sites can influence the framing and scope of policies. 
• Agile delivery methods are being adopted in government. Agile practices—where teams build and release a product incrementally and incorporate feedback along the way—are often part of digital transformation initiatives. They allow agencies to take an iterative approach to improvement, strategically moving toward larger goals. Prototyping outside traditional administrative departments can create a safe space for failure. City-based innovation teams (e.g. San Francisco’s Office of Civic Innovation or Boston’s Office of New Urban Mechanics) can be more open to experimenting and allowing governments to incubate new ideas before pushing them to more traditional agencies. Philanthropies and new funding streams (e.g. the CARES act) can also offer opportunities for experimentation, incubation, and scaling with long-term sponsorship and dedicated financial resources.

At their core, policy prototypes get user feedback more quickly, and, in the long term, can ensure policy design choices meet their intended outcomes. They help practitioners test their assumptions and see how business processes and agency-public interactions come together before committing a large resource investment into policy implementation. The learning that results from every iteration is a powerful input to future efforts and ensures policy goals are fulfilled.