Our Data Methodology

Our system begins by automatically scraping data from highly reputable human rights organizations. Our primary sources include the OVD-Info API (api.repression.info) and Memorial (memopzk.org). Automated scripts visit these sources periodically to gather raw data, including names, case descriptions, locations, and links to original reports.

Raw text from various sources is often unstructured. We use AI models to read and interpret this raw text, extracting key information into a consistent, structured format. The AI is prompted to identify and categorize data points such as arrest dates, criminal charges, case categories, trial statuses, and potential health concerns. It also generates a concise summary for each case in English.

Once structured, we enrich the data further. Location names provided in the source material are converted into precise geographic coordinates (latitude and longitude) using the Nominatim API for OpenStreetMap. This geocoding process is what enables the powerful interactive map feature on our platform, allowing for geographic analysis of arrest locations.

The final, structured, and enriched data for each prisoner is stored in a PostgreSQL database. This robust database serves as the single source of truth for our entire application. A Node.js API, built with Express, provides the data to our Next.js frontend, which then renders the interactive tables, maps, charts, and detail pages you see on this site.