Module Six – Developing and Using the Evidence Inventory

The Evidence Inventory is a well-organized and referenced inventory of documents, processes, and procedures. At a minimum, the Evidence Inventory serves as a resource in four ways:

1. It assists institutions as a tool to document compliance with the standards for accreditation and requirements of affiliation, policies and procedures, and federal compliance requirements. Because the self-study process requires institutions to provide documents and explain and evaluate processes and procedures, a well-developed Evidence Inventory enables institutions to address these in one place for members of the institutional community, the Evaluation Team, and Commissioners to readily access and evaluate.
2. Information found in the Evidence Inventory allows the Self-Study Steering Committee and Working Groups to arrange information in ways that enable them to both support assertions they make within their draft reports and eventually the Self-Study Report itself. In cases where several Criteria are not substantially addressed in the Self-Study Report, appropriate documentation can be supplied in the Evidence Inventory, building confidence that there is sufficient information for appropriate review by the Evaluation Team.
3. The development and use of a well-organized and referenced Evidence Inventory, can bring about efficiencies in the process. For example, when the Evidence Inventory is prepared early in the process, Steering Committee and Working Groups are better able to identify gaps in the documentation that can be discussed and addressed.
4. The development and continuous renewal of the Evidence Inventory empowers institutions to create a long-term resource; once completed, it can serve as a tool for evaluating ongoing compliance with the Commission’s expectations. By developing a well-organized Evidence Inventory, institutions can utilize a tool useful not only for Steering Committee, Working Groups, and peer evaluators, but also for documentation for future reviews.

Figure 1: The Evidence Inventory as a Resource

The Evidence Inventory should be thought of as a referenced resource for both the team and the institution to be used prior to, during, and after the On-Site Evaluation Visit. Documentation in the Evidence Inventory should be directly related to assertions made in the Self-Study Report and constrained to representing compliance with the standards for accreditation, requirements of affiliation, policies and  procedures, and federal compliance requirements. Therefore, the Steering Committee is responsible for ensuring that it does not comprise a “document dump” of items that only tangentially relate to Commission standards or the institution’s chosen priorities.

Phase 1: Gather Initial Evidence

Initial assembly of the evidence can seem daunting at first. Steering Committee and Working Group members should not think of initial attempts to do so as a commentary on an institution’s readiness to engage in self-study or compliance with Commission expectations. Initial steps taken are often beset with concerns about gaps in documentation and how much information is sufficient.

Institutions that have developed an effective, focused, and well-referenced Evidence Inventory first gain a familiarity with the Commission’s expectations and begin collecting documents and descriptions of policies and procedures that might be used in the Evidence Inventory. They do not rely solely on their own knowledge of documentation; instead, they enlist feedback from staff, faculty, and administrators about what documentation might be available. They consult with offices and units on campus, such as financial aid, academic affairs, compliance, registrar and other offices whose staff may know where relevant information is available. Steering Committee members consult frequently with these offices to ensure that identified information sufficiently addresses the Commission’s expectations.

At this point, it is also important to discuss what interface the Steering Committee might use to access information to be found in the Evidence Inventory. Some institutions use a wiki and yet others use commercial software. As the Steering Committee collects documentation it is important to inquire how the information will be warehoused, any technical requirements, and how Working Group members will be given access. Additionally, it will be necessary to ensure that the evidence can be uploaded to the MSCHE portal in the required structure and format.

Phase 2: Refine and Reference

At this point in the process, copious amounts of information may have been identified. It now becomes important to discuss how the information will be limited to documentation that is highly relevant and is not unduly duplicative. Questions to answer during the process could include the following:

• Do documents relate directly to the Standards and their Criteria? For those documents that do not, is it possible to find one or two that more directly relate?
• Do multiple documents relate to one criterion? If so, is it possible for just a few documents to be used to demonstrate compliance?
• Where information is gathered from multiple offices or academic units (e.g., for the purposes of demonstrating the implementation of educational effectiveness assessment process or strategic planning process across an institution), is it possible to summarize the information in the form of a chart or table?
• In cases where Criteria refer to processes and procedures, can the Evidence Inventory contain a process chart or table instead of only lengthier documents? If such charts or tables are available, can they be appropriately referenced to ensure that the team of peer evaluators can understand what they intend to communicate?

The Steering Committee and Working Groups continue to refine the Evidence Inventory in the interest of ensuring that the information provided is both appropriately representative across the institution and comprehensive enough to enable the team of peer evaluators and institutional representatives to access meaningful information. For example, when documenting compliance with Standard V, “Educational Effectiveness Assessment,” some institutions may consider whether the supplied documentation provides sufficient evidence of the assessment of all programs and of institutional outcomes (e.g. those relevant to general education). For Standard VI, “Planning, Resources, and Institutional Improvement,” institutions refining the initial Evidence Inventory may ask if there is sufficient demonstration of a strategic planning approach both institution-wide and for individual units in the documentation.

Phase 3: Use in the Self-Study Process

As the Steering Committee and Working Groups complete reports and move toward an initial and then final draft of the Self-Study Report, they also discuss the effectiveness of the referencing of information found in the Evidence Inventory. Institutions that have developed an effective Evidence Inventory avoid unnecessary acronyms, institutional lingo, or titles that peer evaluators are unfamiliar with. Key terms and acronyms are often spelled out in the form of a glossary or “key institutional phrases and terms” found in the Self-Study Report.

When further refining the Evidence Inventory up to and after the On-Site Evaluation Visit, institutions might ask the following questions in the interest of ensuring focus of the information provided and suitable referencing for those outside the institution:

• When references to the Evidence Inventory are made in the Self-Study Report (or vice versa), are these connections clearly communicated or referenced in the Report itself?
• Is the documentation in the Evidence Inventory complete, appropriately representative and comprehensive so institutional representatives and peer evalutors can effectively and confidently use the information?
• For documents that are necessary but appear complex to an outside reader, is it possible to annotate this information and/or provide excerpts of policies, procedures, reports and other information?
• Does the documentation align with right-to-privacy regulations and respect the privacy of individuals?

It is important to note that not all the Commission’s Standards and their Criteria are equally documentable. Some Criteria or Requirements of Affiliation are easier to document than others. In cases where a Steering Committee or Working Group considers it difficult to provide appropriate documentation, it may want to consider providing information within the Self-Study Report itself.

Phase 4: Upload the Documentation to the Evidence Inventory

Once the Self-Study Report is finished, you must upload the Evidence Inventory to the MSCHE portal.  The Evidence Tab is where you will go to provide supporting documentation for your Self Study Report. Navigate to the appropriate review via the drop-down menu at the top left of the page.  A submission progress bar will track the completion of all relevant criteria for each standard; it does not track the number of documents submitted for each criterion. Please note, this completion percentage is simply a reflection that each section has been acknowledged. It is not an indicator of the quality of the contents.

Some of the features you will find on the portal are as follows:

• The Upload Report tab is where you will upload your Self-Study narrative report.
• The Uploaded Evidence Library provides a summary of all uploaded evidence you have used. Each document you upload as supporting evidence in the Standards and Requirements of Affiliation sections will be added to your Evidence library. This offers you an alternative to uploading a document each time you wish to use it as supporting evidence.
• The Roadmap URLs tab allows you to provide up to five URLs to campus documents, such as Handbooks, Catalogs, and Strategic Plans.  Please do not include URLs in your institution’s Self-Study Report.  However, you may use URLs and external hyperlinks in your Federal Compliance documentation.

Finally, you will use the Additional Documents tab to upload documents requested by the review team prior to or during the on-site visit. After submitting your final Self-Study Report, and upon request by the Review Team, you may submit additional documents up to seven days after the conclusion of the team visit. The team chair will include a list the Additional Documents you need to upload in the team report.

Phase 5: Continuously Refine and Use

After the Evaluation Team has completed its work and the Commission has reaffirmed accreditation, it is advisable that the institution continue to update its documentation periodically. Accreditation is assumed to be ongoing and, should the institution be required to engage in follow-up reporting, or if it completes additional accreditation-related work such as a substantive change application to the Commission, documentation used for the Evidence Inventory can make the process much more efficient and straightforward. In addition, such documentation may be helpful when institutions interact with other accrediting or state regulatory agencies as well as to ensure that the institution need not “reinvent the wheel” when it prepares for its next Self-Study Evaluation with the Commission.

Figure 2. Organizing and Maintaining the Evidence Inventory

1. Include information and technology specialists who have expertise in areas that extend across Working Groups in the self-study process. Regular consultations with information technology specialists, librarians, compliance staff, and others can help Steering Committee and Working Group members to find, reference, and annotate information effectively.
2. Consider organizing an additional Working Group, such as “Working Group 8: Evidence Inventory” whose task is to liaise with other groups so that their documentation efforts are well-sustained.
3. Review the Evidence Inventory frequently and on a regular schedule. Institutions do this in several ways, such as by formally scheduling periodic meetings in which members of the Steering Committee review documentation to be referenced in the Evidence Inventory. Other institutions develop systems where administrative staff or other groups from the Steering Committee separately review the Evidence Inventory and offer feedback.

Tips for Submission of Evidence

• Upload judiciously
• Reference documents directly
• Use a good naming convention
• Think of your team!

Only upload materials needed for the review of the report. This should not be a data dump. Upload evidence that you have specifically referenced in the Self-Study. Secondary information that is not evidence needed for the review of the self-study should not be included.

The team will have adequate time prior to the visit to request additional materials, so you don’t have to worry about uploading everything in case the team may want it.

In the self-study, reference evidence directly by name. This will make it much easier for the team to find the materials.

When naming the evidence you submit, think of a good naming convention that will help your team find materials. Numbering evidence documents in order of how they appear in the self-study can be very useful. It can also be helpful to include the name of the Standard or Requirement of Affiliation in the name of the document.

When putting your self-study together, please think of your team. Your team cannot do an adequate job of reviewing the materials if you upload thousands of pages of evidence. It is too much materials to manage. Treat this as if you’re the one reading it. Like I said earlier, the team will request documents they would like to see, so don’t drown them in thousands of pages “just in case.”

Evidence Inventory Institutional Self-Evaluation Rubric (pdf)