Watcher: only what is worth a look
A self-hosted watcher that follows a set of sources, grades every new item against what you are looking for with an AI model, and ranks the results on one dashboard. It treats everything it reads as untrusted input. Built and run by Agentas on our private cloud infrastructure.
- Status
- In use since June 2026
- Deployment
- One hardened container, internal access only
- Stack
- Python, FastAPI, Claude, Docker
- Source
- Private — walkthrough on request
What it does
Watchlists in plain language
Each watchlist has cheap filters, an alert threshold and a plain-language description of what you want. The AI grades every new item against that description.
AI judges, code does the maths
The model returns sub-scores, warning flags and a short rationale. The final grade is computed in code, so every score can be reproduced.
Only new items cost tokens
Code filters and an exclude list remove noise before any AI call, and a cache means only new or changed items are graded. Token use and cost are recorded for every run.
Built for hostile input
Everything Watcher reads is written by strangers. The AI runs in a sandbox with every tool denied, and external text is capped and fenced as data before it reaches a prompt.
Relevant if you're looking for…
Watcher is a private tool, and we build systems like it for businesses that need to follow a steady stream of documents.
- An AI monitoring tool for sources you pick: each watchlist has cheap filters, an alert threshold and a plain-language description of what you want, and the AI grades every new item against it.
- Prompt injection prevention: everything Watcher reads is written by strangers, so external text is capped and fenced as data before it reaches a prompt.
- An LLM sandbox: the model runs headless with every tool explicitly denied, a lockdown we confirm by probing the real CLI.
- Keeping AI token costs down: code filters and an exclude list remove noise before any AI call, only new or changed items are graded, and token use and cost are recorded for every run.
We build this for clients: AI and LLM integration into your systems · AI automation of manual work
How it works
- 01
Collect
New items from each watchlist's sources.
- 02
Filter
Code rules and exclude terms drop noise before any AI call.
- 03
Grade
A sandboxed AI model judges each item against your intent.
- 04
Rank
Scores computed in code, ranked on the dashboard with the reasoning.
How it's built
Watcher is one Python process: FastAPI serves the dashboard and the API, and a background thread fetches and grades. It ships as one container that runs read-only, as a non-root user, with every Linux capability dropped, and GitHub Actions deploys it on push only after the test suite passes.
Everything it reads is written by strangers, so prompt injection is treated as an everyday risk. The AI runs headless in a sandbox where every tool is explicitly denied, a lockdown we confirm by probing the real CLI, and all external text is capped and fenced as data before it reaches a prompt. At worst a hostile item can skew its own score. It cannot run commands, read files or go online.
Engineering highlights
- Every AI tool explicitly denied, verified against the real CLI
- External text capped and fenced as data before it reaches a prompt
- Signed, HttpOnly session cookies and throttled logins; the app refuses to start without auth
- A content-security policy on every response
- Read-only, non-root container with all capabilities dropped
- Token use and cost recorded for every run
Questions
Is it safe to let an AI model read text written by strangers?
Only with guardrails in place. In Watcher the model has every tool denied and external text is fenced as data, so at worst a hostile item can skew its own score. It cannot run commands, read files or go online.
Who decides the final score, the AI or the code?
The code. The model returns sub-scores, warning flags and a short rationale, and the final grade is computed from them in code, so every score can be reproduced.
Can you build a system like Watcher for our business?
Yes. Any stream of documents can be followed this way: cheap filters first, AI for the judgement, code for the numbers, and a setup that assumes the input is hostile.
Want something like this?
Any stream of documents can be watched this way: cheap filters first, AI for the judgement, code for the numbers, and a sandbox that assumes the input is hostile.
Get in touch