Some projects look simple until you start building them

Just about everything, as it turns out. And that is exactly what made biblija.hr a useful proving ground for a handful of problems that show up on every large content project — only here they are easier to see, because there is nothing else around to obscure them.

This is a write-up of the technical decisions and the mistakes. No polish on the mistakes, because they are the more useful half.

The biblija.hr project is a Bible study portal in Croatian. Every verse comes with the Greek or Hebrew source text, a transliteration, a literal rendering, a word-by-word grammatical breakdown, and a short theological summary.

The idea is modest: let a family or a small group open the Scriptures at home and have beside them what is normally available only to people who studied the biblical languages.

The stack is nopCommerce 4.8 on SQL behind nginx. The New Testament source is the SBL Greek New Testament; the Old Testament uses the Westminster Leningrad Codex with morphology from the Open Scriptures Hebrew Bible. All content is published under CC BY-SA 4.0. The editorial work is volunteer and the site is free. Our interest was the engineering.

With the Bible this is less obvious than it sounds. The underlying text is ancient, but every modern translation is a protected work in its own right. The translator holds copyright as the author of an original work, and it runs for seventy years after their death.

The same applies to the source texts. The Greek New Testament is not one text but a series of critical editions. Nestle–Aland belongs to its publisher and cannot be freely republished; the SBL edition is available under Creative Commons. The difference is one line of configuration and an enormous difference in legal exposure.

The transferable lesson: when the input data is not yours, the inventory of sources and their licences is the first document in the project, not the last.

With open licences it is easy to treat attribution as something that lives in the page footer. It isn't. The obligation applies everywhere the content leaves the system.

• on the page, as a visible element
• in XML and JSON exports, as a field in the document header
• in printable PDFs, in the footer of every page
• in the "copy verse" function, inside the copied text itself

That last one is the one everybody misses. A user copies a verse, pastes it into a slide deck, and the attribution is gone. Technically, that is a licence breach your own system enabled.

Attribution also has to be conditional. Citing the Greek source underneath an Old Testament passage is not merely redundant — it is wrong. Half a line of code, but it is the difference between careful and careless. And it belongs to the module that renders the content, never to the theme: swap the theme and the attribution quietly disappears.

Caching stops being an optimisation. When every page is generated dynamically and carries a security token, it cannot be cached. At two hundred pages that is irrelevant. At thirty thousand, response time directly determines how many URLs a crawler gets through in a day. The fix is to separate the anonymous part of the page, which is static, from the user-specific part, which can be fetched afterwards.

Page weight multiplies by page count. On our pages the same analysis was present in the HTML three times — once rendered, twice inside a JavaScript object used by the interface. Nobody notices that on a single page. Across thirty thousand it is triple the bandwidth and triple the crawl cost.

Sitemaps are not generated per request. Ours is a plain bash script that pulls from SQL, assembles the XML and moves it into the web root atomically. Three details that are easy to get wrong:

• the query returns one row per line of XML, because COPY in text format escapes newlines inside values
• UNION ALL guarantees no ordering, so ORDER BY must live in the export query itself — otherwise the XML declaration can end up in the middle of the file
• write to a temporary name and then rename; mv within one filesystem is atomic, so a crawler never fetches a half-written file

Not everything belongs in the sitemap. A page with no content of its own does not need to be indexed separately. Ours includes every chapter but only those verses that already have a finished analysis; the rest are canonicalised to their chapter. Still reachable for readers, but no longer a burden on the index.

A textbook problem, and it happened anyway.

The site answered on both the bare domain and the www subdomain, with no redirect between them. As far as the protocols are concerned those are two different names, and a search engine is not obliged to connect them — www is technically a subdomain like any other.

The result: each of 31,000 pages existed in two versions. Ranking signals split, crawl budget spent twice on identical content.

Worse, three signals disagreed. The canonical tag pointed at www, the sitemap was published on the bare domain, and no redirect existed at all. The crawler received three contradictory instructions and decided for itself.

The fix was boring and threefold: a 301 in nginx, a single canonical form, and the sitemap on the same host. One detail worth keeping: the HTTP redirect should go straight to the final address, not first to HTTPS and then to the canonical host. Auto-generated configurations routinely do it in two hops.

A related trap: in nopCommerce the canonical tag exists as a mechanism but is only emitted automatically on some page types. Everything else has to add it. We ended up with canonical tags on the verse pages and none on the home page — the worst possible split, since the home page is the one collecting whatever inbound links exist.

Once things were in order, the reporting produced one genuinely unexpected result.

We expected page quality to be the bottleneck: thirty thousand records of identical structure look a lot like machine-generated filler. The opposite turned out to be true — zero pages sitting in the discovery queue, and only five crawled and then rejected. The content passes.

We were also wrong about depth. The crawler found and indexed hundreds of pages before the sitemap was even processed, including verses that take dozens of clicks to reach through the interface. Our assumption that navigation depth was throttling indexation did not survive contact with the data.

What was actually in the way were the technical things: the split domain, thin page titles, and the absence of a page that gathers a whole chapter in one place.

The subject matter is unusual. The problems are not. The same pattern applies to product catalogues, document archives, legal databases — anything with tens of thousands of similar records.

That last point deserves a sentence of its own. Transparency about method is not marketing. It is a precondition for trust, particularly when the content is the kind people approach expecting accuracy.

The Bible turned out to be a rewarding subject precisely because it is demanding: large, highly structured, three languages deep, legally intricate, and read by an audience that notices every error.