API Usage Tracking

[cemde]

Description

Usage tracking is insufficent but important part of evaluation. This PR adds cost tracking infrastructure.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code quality improvement (refactoring, formatting, etc.)

Checklist

Contribution

• I have read the CONTRIBUTING.md guide.
• Commits follow "How to write a good git commit message"

Documentation

• Added/updated docstrings for new/modified functions as instructed CONTRIBUTING.md
• Updated relevant documentation in docs/ (if applicable)
• Tag github issue with this PR (if applicable)

Changelog

• Added entry to CHANGELOG.md under [Unreleased] section
  • Use Added section for new features
  • Use Changed section for modifications to existing functionality
  • Use Fixed section for bug fixes
  • Use Removed section for deprecated/removed features
• OR this is a documentation-only change (no changelog needed)

Example:
- Support for multi-agent tracing (PR:#123)

Architecture (if applicable)

• Core/Interface separation: Changes in maseval/core/ do NOT import from maseval/interface/
• Dependencies: New core dependencies added sparingly; framework integrations go to optional dependencies

Additional Notes

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
maseval
__init__.py
maseval/core
agent.py					199
benchmark.py					474, 1218-1229
model.py					476-477
registry.py					314, 316, 319, 338, 340, 343
usage.py					93, 124-128, 184, 339, 370
maseval/interface
usage.py					75-76, 122
maseval/interface/agents
_cost.py					35-36
camel.py					426, 450-451
langgraph.py					285-288, 315-321
llamaindex.py					243-245
smolagents.py					377
maseval/interface/inference
anthropic.py
google_genai.py
huggingface.py
litellm.py
openai.py					224
Project Total

This report was generated by python-coverage-comment-action