Skip to content

Printing the URL can burn lots of tokens #297

@linkej-autodesk

Description

@linkej-autodesk

Many commands print the page URL whenever the page changed. For very long URLs, this can eat a considerable of the total agent context for little gain. E.g. an option to disable this would be appreciated.

For our application, we put many loading parameters into the URL, which can include e.g. a list of many dozens of files to load, or an 800char access token. This is very useful to e.g. give an agent a repro case just with a single URL.

One example I saw today (take this with a grain of salt, I don't know much about LLMs, maybe they have some deduplication magic)

  • One session with a 143k token context in the end
  • 20 calls to playwright-cli (one open, then eval and run-code), of which 12 printed the URL
  • Just the access token takes 700 tokens according to https://platform.openai.com/tokenizer (don't worry I used an expired token ;) )
  • That's 8400 tokens of 143k, 6%

If you decide to keep this, maybe it can be tuned to happen less often. E.g. it seemed like the first eval after a reload would also print the URL again, even if the eval didn't change anything substantial.

Thank you for this great tool!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions