Measure success

Choose the right things to measure. The government, in general, tends to use metrics that do not accurately measure product success. Things like the number of users, quantity of data sets, and number of applications built do not indicate if these products are meeting user or Department needs. Advana has access to more than 450 DoD data sources, for example, but we know some of that data is not continuously updated or maintained. The implementation team should set success criteria based on measures related to Congress’ ability to self-serve data, answer meaningful questions about the budget and programs, or track if the DoD is meeting NDAA deadlines. These, or other measures, should focus on user needs.

Advana usage metrics should become part of the data that Advana stores and manages. Strict logging and data access metrics should become part of the data that Advana stores and manages. Usage metrics should also be captured inside the enclave to provide the product team with both quantitative and qualitative data to measure Advana user behaviors. Usage metrics would allow a product team to pinpoint areas where users are gaining the most value, and where they are unable to accomplish tasks and rapidly assess system performance issues as they arise.