Not a pile of webpages. A set of libraries.
Most of what matters isn't really a web page — it's a judgment, a standard, a clinical trial, a filing. askFinz keeps each of those as the thing it is, so an answer can point at the right one.
research-and-data/ — works news-and-briefings/ — articles encyclopedia/ — entries industry-and-sector/ — documents markets-and-filings/ — filings medicine-and-clinical-research/ — records products-and-reviews/ — products property-and-construction/ — listings trademarks/ — marks code-repositories/ — repositories books-and-manuals/ — documents course-material/ — courses questions-and-answers/ — threads jobs/ — postings travel/ — listings film-music-and-video/ — titles events-and-recipes/ — entries agriculture/ — records patents/ — publications standards-and-specifications/ — specifications public-tenders/ — notices law-and-legislation/ — documents web-pages/ — pages statistical-indicators/ — indicators earth-observation/ — records software-vulnerabilities/ — disclosures named-entities/ — entities ai-models/ — models insider-transactions/ — transactions general-research/ — records
Read from the running system.
What's held, and what it's for.
30 collections, a page each — filed by what each one holds rather than by where it was found. Filing each page from the open web on the right shelf is the job Sonar, askFinz's own model, is coming to.
Each card is one page: that collection's own size read from the running index, an example of what it is filed to answer, and the fields it keeps on every record. Sizes are per collection and never added together — a single collection's count is a fact about that shelf, not a claim about the index.
The questions are illustrative — written to show what each collection is filed to answer, not run live. The chips are the fields that collection keeps on every record.
We don't scrape a worse copy of something published properly.
A patent is not a web page. Neither is a standard, a clinical trial or a court judgment. Where an organisation publishes its material properly, we read it from them — with the identifiers, dates and status that make it citable — instead of collecting whatever version happens to be lying around on the open web.
Specifications
Read from the standards bodies themselves, so a requirement can be quoted with the document and revision it came from.
Patents & trademarks
Taken from the offices that grant them, keeping the filing office, dates and status rather than a summary of a summary.
Clinical research
Registered studies with their sponsor, phase and status — the trial itself, not an article written about it.
Reading from the source is how these collections get as complete as they are — and it is the part a crawler alone cannot reproduce, no matter how much of the web it reads.
Quality is decided on the way in — and revisited afterwards.
Reading widely is easy; reading widely without filling up on junk is the hard part. Pages are reviewed as they arrive, anything borderline gets a second closer read, and what fails is rejected rather than kept and filtered later. These are live figures.
It also improves what is already held. As the software that reads and files pages gets better, it goes back over material indexed months ago and re-checks it — so a page read early benefits from every improvement made since, without anyone asking for it and without the page being fetched again. The index gets better with age rather than staler.
Read live from the running system. Shown as rates rather than totals — the index is still growing, and a rate tells you how it is built rather than how far along it is.
Fetched once, when it was first read.
reading…
- Read and storedthe page enters the index
- Cleaner textmenus, banners and filler stripped more accurately
- Structure kepttables and headings survive as tables and headings
- Filed by kindrecognised as the sort of thing it is, not a loose page
Nothing above sends a single new request to the publisher. Every improvement is made against the copy we already hold — which is why a page read early keeps getting better, and why the index improves with age rather than going stale.
The same thing, recognised on every shelf.
Nearly everything worth indexing already has an official reference number. A company has a registration number. A paper has a publication reference. A court case has a case number. A package, a film, a recording, a trademark, a tender, a statistical series — each is published with an identifier by the body that holds it. Almost every search engine throws that away on the way in and keeps the title.
We keep it. Forty-two different kinds of reference number, across nineteen of the collections, stored exactly as the source published them. It sounds like bookkeeping. It is the difference between two records being matched and two records being guessed at, because names change, names repeat, and one thing is written down differently by everyone who writes it down.
ACME HOLDINGS PUBLIC LIMITED COMPANY
Annual report and accounts
Acme Holdings
Share purchase disclosed by an officer
ACME HOLDINGS P.L.C.
Published notice
One lookup. All three carry the company number their own source published, so they are recognised as the same company — whatever each one chose to call it.
The identifier is not something we invent. It is the one the source already publishes alongside the record, and which most search throws away on the way in. Keeping it is what lets two records be checked against each other rather than eyeballed.
Companies and markets
Filings, ownership and the share dealings of directors, each kept under the registration number the registry itself uses — not a spelling of the name.
Research and data
Papers, the datasets behind them and the institutions that produced them, under the publication references publishers already print.
Law and legislation
Judgments and the legislation they turn on, kept under the case and document numbers the courts assign rather than under a case name.
Software
Packages, the repositories they are built from and the advisories written against them, under the package and advisory references the ecosystem already uses.
Public spending
Tenders, awards and trademarks, under the notice and registration numbers the issuing body published them with.
Media and measurement
Films, recordings, statistical series and recorded natural events, each under the catalogue or series reference its own registry assigns.
Being straight about how far this reaches: recording the reference number is done, and it is the hard part to retrofit. Having the same subject sitting in two different collections so there is something to join to is still the exception rather than the rule, and it grows as the collections fill. And where a source genuinely publishes no identifier — a news story mentions several companies and formally names none of them — we leave the record unlinked rather than guess a connection that might be wrong.
Filed by what it is
A court judgment is kept as a judgment, with its jurisdiction and the cases around it — not as a page that happens to contain legal words. That is what lets an answer cite the right thing.
Read from the source
Where an organisation publishes its own material properly, we read it from there rather than scraping a worse copy off the open web.
Growing where it's used
Collections expand toward what people actually ask about. Because the index is ours, adding a new one is a decision we can simply make.
The shelves grow where people look.
Point us at yours.
Collections expand toward what readers actually ask about, and because the index is ours, adding to one is a decision we can simply make. If a source you rely on isn't here, tell us — and if the site is yours, you can have it read directly with priority rather than waiting your turn.