Module Six – Developing and Using the Evidence Inventory

The Evidence Inventory is a helpful, organizational tool that allows an institution to arrange existing institutional documentation gathered for the self-study by Standard, Criterion, and Requirement of Affiliation. Institutions utilize various strategies to initially populate and then refine the Evidence Inventory. One strategy is to assign a member of the Steering Committee to liaise with key personnel on campus who can help with the documentation process. Another strategy is to organize a specific Working Group for this task. Whatever the strategy, the final version of the Evidence Inventory must be in a format accessible to key individuals and groups, such as Steering Committee members and institutional staff, as well as able to be uploaded to the secure MSCHE (Middle States Commission on Higher Education) portal for the Evaluation Team, the Committee on Evaluation Reports, and the Commission.

To print this module, please expand any or all sections.

Purposes of the Evidence Inventory (Module 6, Section 1)

The Evidence Inventory is a well-organized and referenced repository of documents, processes, and procedures. At a minimum, the Evidence Inventory serves as a resource in four ways:

  1. It assists institutions as a tool to document compliance with:
    • the Standards for Accreditation and Requirements of Affiliation,
    • Commission policies and procedures, and
    • Accreditation-related Federal compliance requirements.

Because the self-study process requires institutions to provide documents and explain and evaluate processes and procedures, a well-developed Evidence Inventory enables institutions to address these in one place for members of the institutional community, the Evaluation Team, and Commissioners to readily access and evaluate.

  1. The Evidence Inventory aids the Self-Study Steering Committee and Working Groups in arranging information in ways that enable them to both support assertions they make within their draft reports and eventually the Self-Study Report itself. In cases where several criteria are not substantially addressed in the Self-Study Report, appropriate documentation can be supplied in the Evidence Inventory, building confidence that there is sufficient information for appropriate review by the Evaluation Team.
  2. The development and use of a well-organized and referenced Evidence Inventory can bring about efficiencies in the process. For example, when the Evidence Inventory is prepared early in the process, Steering Committee and Working Groups are better able to identify gaps in the documentation that can be discussed and addressed.
  3. The development and continuous renewal of the Evidence Inventory empowers institutions to create a long-term resource; once completed, it can serve as a tool for evaluating ongoing compliance with the Commission’s expectations. By developing a well-organized Evidence Inventory, institutions can utilize a tool useful not only for Steering Committee, Working Groups, and peer evaluators, but also for documentation for future reviews as well as serve as a helpful resource in the institution’s ongoing planning activities.

Procedure for Organizing and Maintaining an Evidence Inventory (Module 6, Section 2)

The Evidence Inventory should be thought of as a referenced resource for both the team and the institution to be used prior to, during, and after the On-Site Evaluation Visit. Documentation in the Evidence Inventory should be directly related to assertions made in the Self-Study Report and constrained to representing compliance with the standards for accreditation, requirements of affiliation, policies and procedures, and federal compliance requirements. Therefore, the Steering Committee is responsible for ensuring that it does not comprise a “document dump” of items that only tangentially relate to Commission standards or the institution’s chosen priorities.

Phase 1: Gather Initial Evidence

Initial assembly of the evidence can seem daunting at first but can be useful in developing strategies for further development. Steering Committee and Working Group members should not think of initial attempts to do so as a commentary on an institution’s readiness to engage in self-study or a predictor of future success. Initial steps taken are often beset with concerns about gaps in documentation and how much information is sufficient. Be not afraid! These initial steps are in many ways an assessment used to identify the next steps.

Institutions that have developed an effective, focused, and well-referenced Evidence Inventory first gain a familiarity with the Commission’s expectations and begin collecting documents and descriptions of policies and procedures that might be used in the Evidence Inventory. They do not rely solely on their own knowledge of documentation because this can be overly ambitious; instead, they enlist feedback from staff, faculty, and administrators about what documentation might be available. They consult with offices and units on campus, such as financial aid, academic affairs, compliance, registrar, and other offices whose staff may have expertise related to the standard and know where relevant information is available. Steering Committee members consult frequently with these offices to ensure that identified information sufficiently and appropriately addresses the Commission’s expectations.

At this point, it is also important to discuss what interface the Steering Committee might use to access information to be found in the Evidence Inventory. Some institutions use internal platforms and some use external or commercial software. As the Steering Committee collects documentation it is important to inquire how the information will be warehoused, any technical requirements, and how Working Group members will be given access. Additionally, it is necessary to ensure that the evidence can be uploaded to the MSCHE portal in the required structure and format.

Phase 2: Refine and Reference

At this point in the process, copious amounts of information may have been identified. It now becomes important to discuss how the information will be limited to documentation that is highly relevant and is not unduly duplicative. Questions to answer during the process could include the following:

  • Do documents relate directly to the standards and their criteria? For those documents that do not, is it possible to find one or two that more directly relate?
  • Do multiple documents relate to one criterion? If so, is it possible for just a few documents to be used to demonstrate compliance?
  • Where information is gathered from multiple offices or academic units (e.g., for the purposes of demonstrating the implementation of educational effectiveness assessment process or strategic planning process across an institution), is it possible to summarize the information in the form of a chart or table?
  • Is the information provided sufficiently representative to build an accurate snapshot of the status of the institution’s compliance with the standards and their associated criteria?
  • Are relevant and appropriate data analyzed and results or recommendations provided?
  • Are the analyzed data presented in ways that lead one to believe results were regularly considered and used by stakeholders in the interest of continuous improvement?
  • In cases where criteria refer to processes and procedures, can the Evidence Inventory contain a process chart or table instead of only lengthier documents? If such charts or tables are available, can they be appropriately referenced to ensure that the team of peer evaluators can understand what they intend to communicate?

The Steering Committee and Working Groups continue to refine the Evidence Inventory in the interest of ensuring that the information provided is both appropriately representative across the institution and comprehensive enough to enable the team of peer evaluators and institutional representatives to access meaningful information. Please see the examples below:

  • Standard I: Mission and Goals—An institution’s stakeholders may ask if consultants were part of a mission review process to learn if there documents or evaluation results were used to revision the institution’s mission and goals the last time such an event occurred.
  • Standard II: Ethics and Integrity—Institutions addressing documentation of accreditation-relevant policies and procedures inquire within their institutions if there is a policy committee, internal control office, or “policy on policies” that regularly review, evaluate, and revise policies or create new ones.
  • Standard III: Design and Delivery of the Student Learning Experience--Several institutions explore their current assessment processes to see if there are opportunities for efficiency. Past examples of success have included the aggregation, presentation, and use of course or teaching evaluation results, program review results, and reports from specialized accreditors.
  • Standard IV: Support of the Student Experience—some institutions work with their institutional research units to learn if institutional and unit information is regularly compiled and distributed and if such data are broken down by key sub-populations.
  • Standard V: Educational Effectiveness Assessment – some institutions may consider whether the supplied documentation provides sufficient evidence of the assessment of all programs and of institutional outcomes (e.g., those relevant to general education).
  • Standard VI: Planning, Resources, and Institutional Improvement – institutions refining the initial Evidence Inventory may ask if there is sufficient demonstration of a strategic planning approach both institution-wide and for individual units in the documentation.
  • Standard VII: Governance, Leadership, and Administration: Institutions wanting to document the oversight role of a governing body may ask if there is a way to systematically evaluate governing board meeting minutes to discover emerging themes that might track with Commission Standards and their criteria and can be useful for the working group assigned to the standard.

Phase 3: Use in the Self-Study Process

As the Steering Committee and Working Groups complete reports and move toward an initial and then final draft of the Self-Study Report, they also discuss the effectiveness of the referencing of information found in the Evidence Inventory. Institutions that have developed an effective Evidence Inventory avoid unnecessary acronyms, institutional lingo, or titles that peer evaluators are unfamiliar with. Key terms and acronyms are often spelled out in the form of a glossary, or “key institutional phrases and terms” found in the Self-Study Report. When further refining the Evidence Inventory up to and after the On-Site Evaluation Visit, institutions might ask the following questions in the interest of ensuring focus of the information provided and suitable referencing for those outside the institution:

  • When references to the Evidence Inventory are made in the Self-Study Report (or vice versa), are these connections clearly communicated or referenced in the Report itself?
  • Is the documentation in the Evidence Inventory complete, appropriately representative, and comprehensive so institutional representatives and peer evaluators can effectively and confidently use the information?
  • For documents that are necessary but appear complex to an outside reader, is it possible to annotate this information and/or provide excerpts of policies, procedures, reports, and other information?
  • Does the documentation align with right-to-privacy regulations and respect the privacy of individuals?
  • In cases where a document label is actually a file name, is it clearly understood what information can be accessed by clicking on the file name?

It is important to note that not all the Commission’s Standards and their criteria are equally documentable. Some criteria or Requirements of Affiliation are easier to document than others. In cases where a Steering Committee or Working Group considers it difficult to provide appropriate documentation, it may want to consider providing information within the Self-Study Report itself.

Phase 4: Upload the Documentation to the Evidence Inventory

Once the Self-Study Report is finished, you must upload the Evidence Inventory to the MSCHE portal. The Evidence Tab is where you will go to provide supporting documentation for your Self Study Report.

Navigate to the appropriate review via the drop-down menu at the top left of the page. A submission progress bar will track the completion of all relevant criteria for each standard; it does not track the number of documents submitted for each criterion. Please note, this completion percentage is simply a reflection that each section has been acknowledged. It is not an indicator of the quality of the contents. Some of the features you will find on the portal are as follows:

  • The Upload Report tab is where you will upload your Self-Study Report.
  • The Uploaded Evidence Library provides a summary of all uploaded evidence you have used. Each document you upload as supporting evidence in the Standards for Accreditation and Requirements of Affiliation sections will be added to your Evidence library. This offers you an alternative to uploading a document each time you wish to use it as supporting evidence.
  • The Evidence Inventory URLs tab allows you to provide up to five URLs to campus documents, such as Handbooks, Catalogs, and Strategic Plans. Please do not include URLs in your institution’s Self-Study Report. However, you may use URLs and external hyperlinks in your Federal Compliance documentation.

Finally, you will use the Additional Documents tab to upload documents requested by the review team prior to or during the on-site visit. After submitting your final Self-Study Report, and upon request by the Review Team, you may submit additional documents up to seven days after the conclusion of the team visit. The team chair will include a list of the Additional Documents you need to upload in the team report.

Tips for Submission of Evidence (Module 6, Section 3)

Below are some important things to remember when selecting and submitting documentation for the Evidence Inventory.

  • Only upload materials needed for the review of the report. This should not be a data dump.
  • Upload evidence that you have specifically referenced in the Self-Study Report. Secondary information that is not evidence needed for the review of the Self-Study Report should not be included.
  • Data should be analyzed, and results summarized. Indicate how the results informed decision making, changes, etc.
  • In the Self-Study Report, reference evidence directly by name. This will make it much easier for the team to find the materials.
  • Use a good naming convention. Numbering evidence documents in order of how they appear in the Self-Study Report can be especially useful. It can also be helpful to include the name of the Standard or Requirement of Affiliation in the name of the document.
  • The team will have adequate time prior to the visit to request additional materials, so you do not have to worry about uploading everything in case the team may want it.
  • Review the evidence from the team members’ perspective. Your team cannot do an adequate job of reviewing the materials if you upload thousands of pages of evidence. Treat this as if you are the one reading it. Please do not drown them in thousands of pages “just in case.”

Keys to Success (Module 6, Section 4)

  • Include information and technology specialists who have expertise in areas that extend across Working Groups in the self-study process. Regular consultations with information technology specialists, librarians, compliance staff, and others can help Steering Committee and Working Group members to find, reference, and annotate information effectively.
  • Consider organizing an additional Working Group, such as “Working Group 8: Evidence Inventory” whose task is to liaise with other groups so that their documentation efforts are well-sustained.
  • Review the Evidence Inventory frequently and on a regular schedule. Institutions do this in several ways, such as by formally scheduling periodic meetings in which members of the Steering Committee review documentation to be referenced in the Evidence Inventory. Other institutions develop systems where administrative staff or other groups from the Steering Committee separately review the Evidence Inventory and offer feedback.
  • Show clear connections between the narrative and the documentation in the Evidence Inventory.

Resource Toolbox (Module 6, Section 5)

Frequently Asked Questions

Question Answer
Should we submit evidence since the last self-study? If there is a Commission action directing the institution to emphasize any Standards in the next self-study, then submit evidence that is related to the prior self-study recommendation(s). Otherwise, submit evidence for the previous 3-5 years.
What if our financial statements will not be ready by the deadline to submit the Evidence Inventory? Please submit the latest available financial statements. If new financial statements become available prior to the self-study evaluation visit, please discuss with the Team Chair and upload to the MSCHE portal.
Should we include assessment reports for all academic programs? Every degree program should demonstrate educational goals and a program assessment plan. However, depending on the number of academic degree programs, the institutions could provide a sample of reports from all degree levels and types of programs including both specialized and non-accredited programs. The samples should be appropriately representative and comprehensive.
Can we upload a document to more than one criterion? Yes, if it is relevant to multiple criteria. See next question.
Can large documents, such as a Faculty Handbook, which is referenced in multiple places in a self-study, be uploaded only once and then referenced with page numbers? Yes, the document can be uploaded once and then referenced as appropriate. Institutions can also choose to include a link to the Faculty Handbook as one of the five URLs allowed in the Evidence Inventory.

 

If a document is referenced by more than one standard, may it be uploaded just once? Yes, each uploaded document becomes part of the Evidence Library. You may choose a document from the Evidence Library and assign it to another criterion.

 

If a document is uploaded and then the institution wants to remove it, must it ask Middle States to remove it? Documents can be removed from the sections on the Standards and Requirements of Affiliation as long as the self-study has not been submitted yet. However, once the self-study has been submitted no deletions are permitted.
Can a submitted document be overwritten by giving it the same name? While this can be done, it may result in confusion, and it is not advisable.
Do we need to provide confidential information such as the president’s evaluation? Personally identifiable information should not be included in the Evidence Inventory. In the example cited, an institution should only provide the evaluation tool and process for evaluating the president.
If we publish our Self-Study Report on our website, do we need to make the Evidence Inventory public as well? It is up to the institution to decide whether to publish the self-study report. Even if you do make it public, you do not need to make the Evidence Inventory public as well.
Is it accurate that there are to be no hyperlinks in the Self-Study Report, and institutions are limited to the five in the Evidence Inventory? That is correct.

 

Is it permissible for pieces of evidence to be stored in a password protected website?

 

No. Documents that are in an online format document should be assigned one of the URLS or upload it to the Evidence Inventory. It should not be password restricted for the readers.
At the end of each chapter (by standard), is it required that the items in the Evidence Inventory be listed? No. It is often done, but not required.

 

Does MSCHE have any samples or recommendations for naming conventions and citations in the text?

 

MSCHE does not endorse any one approach. Different institutions do it differently depending on how they set up their self-study. The most important thing is for an institution to think about the experience of the team and make that experience as user-friendly as possible. It is best to keep file names short and descriptive.

 

 

Evidence Inventory Institutional Self-Evaluation Rubric (pdf)

Disclaimer: The material provided in this guide was developed to provide clarity for the self-study process. Commission Policy and Procedures will govern in the case of a conflict with this material. For any questions about an institution’s accreditation status or for additional information about MSCHE’s standards for accreditation, requirements of affiliation   policies, and procedures, you should contact MSCHE staff. This material is not intended as a substitute for professional advice from MSCHE staff and use of the material does not guarantee any specific accreditation outcome.