Research · Round 1

The AI astrology consistency benchmark.

Astrology apps are compared on vibes and app-store stars. Two things about them are actually measurable: whether they agree with a declared chart reference, and whether they contradict themselves. This is the method for testing both, published before the test is run.

Pre-registered✦Synthetic birth data✦Open rubric
Read this first: who runs this test

This benchmark is designed, run and published by Vedikks, and Vedikks is one of the products being tested. That is a conflict of interest, and no methodology statement makes it go away. What we can do instead is make the test checkable: the birth data, the prompts, the scoring rubric and the reference key are all published below, before the run, so anyone can repeat it and get their own numbers.

We will not place Vedikks first unless the raw scores put it first. If they do not, the table will say so. Every product tested is invited to submit corrections, and corrections that hold up will be applied with a dated note.

Status: protocol published, Round 1 not yet run

Published 25 August 2026. The results tables below are deliberately empty. They will be filled only with observations that were actually made, each carrying the date and conditions of the run. Nothing on this page is a claim about any product's performance yet.

Why this test and not another

Almost every "best astrology app" list ranks products on interface, price and how the writer felt. Those are real, but they are not testable, and the reader has no way to check them. Two properties of an AI astrology product are testable by anyone with a browser and an hour.

The first is arithmetic. A birth chart is astronomy. Given a date, a time, a place and a stated ayanamsa, there is a right answer for where the Moon was, and apps either match it or they do not. Where they differ, the reason is almost always a declared convention — a different ayanamsa, mean versus true node, a timezone assumption — and naming that reason is more useful than calling anyone wrong.

The second is consistency. If an app gives you one answer today and the opposite answer in a fresh session tomorrow, it does not matter how elegant either answer was. And if you tell an app something about your life and it later contradicts it, that is the specific failure that makes people close astrology apps for good.

Interpretation quality — whether a reading is good — is deliberately not scored. It is real, and it is not measurable by a rubric without smuggling in the tester's taste.

The protocol

Birth data

Three synthetic charts are used. They belong to nobody. They are published in full, with expected values, on our method page and as reference-charts.json.

CaseLocal birth date and timeUTC offsetPlaceWhy this case
REF-A1 January 2000, 12:00:00+05:30New DelhiOrdinary case, nothing near a boundary. Baseline.
REF-B15 June 1990, 05:45:00+05:30MumbaiSun at Taurus 29°56'. Four arcminutes from a sign change; separates ayanamsa conventions.
REF-C9 November 1975, 23:10:00+05:30ChennaiLate-evening birth, southern latitude, older date. Tests time and house handling.

The reference values are computed with the Swiss Ephemeris library in Moshier mode, sidereal, Lahiri ayanamsa, mean node. That is a declared reference, not a claim to be the only correct convention. An app that uses the true node or a different ayanamsa is scored as differs — convention, not as wrong, provided it states its convention somewhere a user can find.

What gets scored

#CriterionHow it is observedScale
C1Ascendant agreementEnter REF-A/B/C, read the reported lagna sign and degreematch / differs — convention / differs — unexplained
C2Moon sign and nakshatra agreementSame, reading Moon sign, nakshatra and padamatch / differs — convention / differs — unexplained
C3Dasha agreementCompare the reported running mahadasha and its start datematch / differs / not offered
C4Convention transparencyIs the ayanamsa and node type stated anywhere public?stated / findable in support docs / not stated
C5Self-consistencyAsk Q1–Q6 in session 1, repeat verbatim in a fresh session 24h later0–6 answers materially unchanged
C6Contradiction rate after a stated factState the fact in Q7, then ask Q8–Q10 which invite a contradictioncontradictions in 0–3 of 3
C7Data controlsCan a user view, correct, export or delete stored context in-product?count of 4 present
C8Signup frictionWhat is required before a chart appears: nothing / email / phone OTP / paymentobserved, with date
C9Price to reach a first real answerObserved and dated; free tiers change oftenobserved, with date
C10Stated limitsDoes the product say anywhere that it is not medical, legal or financial advice?present / absent

The prompt set

Every product gets these ten, in this order, in one session, then Q1–Q6 again verbatim in a fresh session at least 24 hours later.

  1. What is my ascendant and which nakshatra is my Moon in?
  2. Which mahadasha am I running right now, and when did it start?
  3. What does my 10th house say about the kind of work that suits me?
  4. Is there anything difficult in my chart around money?
  5. What is the single most important thing in this chart?
  6. How is the next twelve months looking overall?
  7. Just so you know: I am not looking to get married. I want the next two years to be about my career.
  8. How is the next year looking for me?
  9. Is there a good time for a big life event coming up?
  10. What should I be planning for in the first half of next year?

Q7 is the whole point of Q8–Q10. A product that answers Q8 with wedding timing has contradicted something the user stated four messages earlier, in the same session. A product that still does it in the next session has no continuity at all. Both are recorded, separately.

Which products

Between six and ten publicly accessible products. The final list is fixed on the day of the run and will be recorded here with the reason each was included, drawn from: products that actually appear in Indian search results for AI astrology queries on the run date; the large Indian incumbents; and Vedikks. Products whose primary tradition is Western rather than Vedic will be labelled as such and scored only on C5–C10, since C1–C3 would compare them against a reference their system does not use.

No product is contacted before the run. Every account is created as an ordinary user, at ordinary prices, with no press access.

Results

Round 1 has not been run. This table publishes empty so that the columns cannot be chosen after the fact to suit the outcome.

ProductC1–C3 calculationC4 transparencyC5 consistencyC6 contradictionsC7 data controlsObserved on
Fixed on run daypendingpendingpendingpendingpendingpending

When Round 1 publishes, the raw per-question observations go up as a downloadable file alongside this page, not just the summary scores. A benchmark that publishes only its own aggregate is asking to be trusted rather than checked.

Limitations, stated up front

  • We are an interested party. Vedikks designed this test and is in it. Read every Vedikks row with that in mind, and repeat the run yourself if it matters to you.
  • The reference is a convention, not a truth. Lahiri and mean node are defensible and widely used. They are not the only defensible choice, and C1–C3 measure agreement with a declared standard, not correctness.
  • Language models are not deterministic. Two runs of the same product can differ for reasons that have nothing to do with the product's design. That is precisely why C5 measures repeats rather than one-shot quality, and why single-point differences are not treated as findings.
  • Three charts is a small sample. It is enough to expose convention differences and gross calculation errors. It is not enough to characterise a product's accuracy in general.
  • Products change. Every observation carries a date. A score from August tells you about August.
  • Interpretation quality is not measured. It matters more than anything scored here, and it is not measurable this way.

Corrections

If you build one of the products in this test and think an observation is wrong, write to support@vedikks.com with the case, the question and what you believe the correct observation is. If it holds up, the table changes and a dated note goes in the change log below saying what changed and who raised it. We would rather be corrected in public than be quietly wrong.

The same applies to the reference charts themselves. They are reproducible from the code on our method page. If they do not reproduce, that is our defect.

  • 2026-08-25 — Protocol, prompt set, rubric and reference key published. No results yet.

Test us with REF-B

The boundary case is the interesting one. Enter 15 June 1990, 05:45, Mumbai, and check the Sun against the published value before you believe anything else on this site.

Open the chart tool