SafetyPage

Government · Government · NIST

The TEVV-Athlon Framework for Evaluating AI Systems - ipd

ReadSource at nvlpubs.nist.gov ↗Read herePublic domain (US government work)

Outline · 41 pages

  1. Introductionp. 9
  2. The TEVV-Athlon Frameworkp. 10
  3. Articulate & Organizep. 11
  4. Define & Constructp. 13
  5. Apply & Measurep. 14
  6. Synthesize & Interrogatep. 15
  7. Example TEVV-Athlon: The Query-Violation Problemp. 15
  8. Articulate & Organizep. 15
  9. Define & Constructp. 18
  10. Apply & Measurep. 18
  11. Synthesize & Interrogatep. 20
  12. General Guidance for a TEVV-Athlonp. 20
  13. Gathering Appropriate Expertisep. 20
  14. Checking Basic TEVV-Athlon Requirements, Resources, and Materialsp. 21
  15. Composing a TEVV Toolboxp. 21
  16. Checking AI Software and Outputs with Model Testingp. 21
  17. Testing in More Realistic Settingsp. 22
  18. Applying Best Practices for Scientific Measurementp. 23
  19. Validating Measurementp. 23
  20. Beware of Goodhart's Lawp. 24
  21. Conclusion and Discussionp. 24