Setting the Stage

Measuring poverty and household wealth in low- and middle-income countries is challenging. Traditional surveys, like those from the Demographic and Health Surveys (DHS), provide reliable data but are expensive, infrequent, and often miss remote or fast-changing areas. Researchers have started using satellite images to spot signs of wealth, such as roads or buildings, but these visuals alone can’t capture cultural, historical, or social factors. With advances in AI, including large language models (LLMs) like GPT, we can now tap into text from the internet or AI-generated descriptions to add context. This paper explores whether combining satellite images (visual data) with text (from AI “memory” or web searches) can create a more complete picture of poverty, potentially revealing a shared “ideal” representation of wealth across these data types—drawing from ideas like the Platonic Representation Hypothesis.

What the Authors Did

The authors analyzed data from more than 60,000 DHS neighborhoods across Africa, spanning 1990 to 2020, using the International Wealth Index (IWI) as a measure of household wealth. For each location, the authors paired high-resolution satellite images (from Landsat, showing physical features like infrastructure) with two types of text: (1) descriptions generated by LLMs based only on the location and year, drawing from the model’s built-in knowledge; and (2) real-world text retrieved by an AI “search agent” that queries the web (e.g., Wikipedia or news) for historical and economic details. The authors built five prediction systems: one using just images, one using LLM text, one using agent-searched text, one fusing images and text into a shared code, and an ensemble combining all. The authors tested these on different data splits (random, out-of-country, out-of-time) to check robustness. Results showed that blending images with LLM text boosted prediction accuracy (e.g., explaining 77% of wealth variation vs. 63% for images alone), with LLM knowledge outperforming web-searched text. Embeddings (compact AI representations) from images and text showed moderate overlap (similarity around 0.60), suggesting some shared underlying structure.

Why the Results Matters

This work advances poverty mapping by making it more accurate and scalable, helping policymakers target aid in underserved African regions without relying solely on costly surveys. It highlights how AI can bridge gaps in data by fusing visuals and text, with LLM “memory” proving surprisingly effective for generalization across countries and time periods. While web-searched text added minor unique insights (weak support for “agent-induced novelty”), the findings broadly align with the idea of unified representations in AI. The authors released a large multimodal dataset on Hugging Face (with ~60,000 entries including images, texts, and IWI labels) to enable further research in AI for social good, such as fairer models or causal analysis, ultimately supporting global efforts to reduce inequality.