org-notes/equinix/design/permissions-migration/test-plan.org

Testing IAM-Runtime checks for Metal API

• What's changed
• Stages of testing
• Monitoring
• Handling broken cases

What's changed

Stages of testing

• Initial Canary

  • Run terraform against internal canary URL

• Slow roll to production

  • Watch for errors

• In-production warn mode

  • Observe for discrepancies between cancancan/iam-runtime
• Runtime winning mode
• Completed

Monitoring

• Trace attributes that are relevant
• Dashboards
• Create dashboard around cancancan disagreements
• Create dashboard where resource was not metal org/project/user

Handling broken cases