--- title: "Cultural Interpretability: Bengali Literature Case Study" author: "Rantideb Howlader" date: "2026-05-24T00:00:00.000Z" canonical_url: "https://www.ranti.dev/blog/cultural-interpretability-case-study-bengali-literature" license: "CC-BY-4.0" --- ## Introduction: Why Bengali Literature and Why Now Bengali is the seventh most spoken language in the world. Its literary tradition spans over a thousand years, from the Charyapada poems of the 8th century to the contemporary fiction of writers like Jhumpa Lahiri and Neel Mukherjee. This tradition produced Asia's first Nobel laureate in literature (Rabindranath Tagore, 1913) and shaped the cultural identity of over 300 million people across Bangladesh and the Indian state of West Bengal. Yet when we examine how large language models represent Bengali literary traditions internally, we find something troubling. The models know less, remember less precisely, and organize their knowledge less richly for Bengali literature than for Western canonical traditions of comparable historical significance. This is not a failure of the models. It is a reflection of the training data they consumed, which overwhelmingly represents English-language cultural production. This case study applies cultural interpretability methods to make this representational asymmetry visible, measurable, and discussable. It is not an accusation. It is a diagnostic. And it comes with a full reproducible research notebook so that other scholars can verify the findings, challenge the methodology, and extend the analysis to their own cultural traditions. ## Research Design ### Research Questions This case study asks three questions: 1. How does the internal representational density for Bengali literary referents compare to Western canonical referents in a language model? 2. Which specific Bengali authors and works are most affected by representational sparsity? 3. What patterns of confabulation (factual errors in model completions) appear for Bengali literary referents? ### Method Overview ```mermaid flowchart TD A[Define Referents] --> B[Construct Prompts] B --> C[Extract Activations] C --> D[Encode with SAE] D --> E[Measure Density] D --> F[Analyze Features] D --> G[Test Completions] E --> H[Compare Groups] F --> H G --> H H --> I[Interpret Findings] ``` We use three metrics from the cultural interpretability framework: 1. **Mnemonic density**: How many SAE features activate for a given referent? Higher density means richer internal representation. 2. **Feature specificity**: Are the activated features specific to this referent, or are they generic features shared across many referents? 3. **Completion accuracy**: When the model generates text about a referent, how factually accurate is it? ### Model and Tools - Model: Pythia-410M (open weights, well-studied, SAEs available) - SAE: Pretrained SAE from SAELens (layer 6, 32768 features) - Hardware: Single NVIDIA A100 40GB - Software: Python 3.11, PyTorch 2.2, transformers 4.38, SAELens 3.2 We chose Pythia-410M because it is small enough to analyze thoroughly on a single GPU, large enough to have meaningful internal representations, and has high-quality pretrained SAEs available. The methods scale to larger models (Pythia-6.9B, Gemma 2) with proportionally more compute. ## Defining Cultural Referents The first step in any cultural interpretability study is defining your referents: the cultural entities whose model representations you want to examine. This is an interpretive choice that must be justified. ### Bengali Literary Referents We selected 12 Bengali literary referents spanning different periods, genres, and levels of international recognition: ```python bengali_referents = { # Classical period "chandidas": { "full_name": "Baru Chandidas", "period": "14th-15th century", "known_for": "Srikrishnakirtan, Vaishnava poetry", }, "bharatchandra": { "full_name": "Bharatchandra Ray", "period": "18th century", "known_for": "Annadamangal, court poetry", }, # Bengal Renaissance "madhusudan": { "full_name": "Michael Madhusudan Dutt", "period": "19th century", "known_for": "Meghnadbadh Kavya, Bengali sonnets", }, "bankim": { "full_name": "Bankim Chandra Chattopadhyay", "period": "19th century", "known_for": "Anandamath, Vande Mataram", }, "tagore": { "full_name": "Rabindranath Tagore", "period": "19th-20th century", "known_for": "Gitanjali, Nobel Prize 1913", }, "nazrul": { "full_name": "Kazi Nazrul Islam", "period": "20th century", "known_for": "Bidrohi, rebel poetry", }, # Modern period "jibanananda": { "full_name": "Jibanananda Das", "period": "20th century", "known_for": "Banalata Sen, modernist poetry", }, "manik": { "full_name": "Manik Bandyopadhyay", "period": "20th century", "known_for": "Padma Nadir Majhi, realist fiction", }, "sunil": { "full_name": "Sunil Gangopadhyay", "period": "20th century", "known_for": "Sei Somoy, historical fiction", }, "mahashweta": { "full_name": "Mahashweta Devi", "period": "20th century", "known_for": "Hajar Churashir Maa, activist literature", }, "humayun": { "full_name": "Humayun Ahmed", "period": "20th-21st century", "known_for": "Himu series, Bangladeshi fiction", }, "taslima": { "full_name": "Taslima Nasrin", "period": "20th-21st century", "known_for": "Lajja, feminist writing", }, } ``` ### Western Canonical Referents (Baseline) For comparison, we selected 12 Western literary referents matched by historical period and canonical status: ```python western_referents = { "chaucer": {"period": "14th century", "known_for": "Canterbury Tales"}, "shakespeare": {"period": "16th-17th century", "known_for": "Hamlet, sonnets"}, "milton": {"period": "17th century", "known_for": "Paradise Lost"}, "austen": {"period": "19th century", "known_for": "Pride and Prejudice"}, "dickens": {"period": "19th century", "known_for": "Great Expectations"}, "eliot_george": {"period": "19th century", "known_for": "Middlemarch"}, "woolf": {"period": "20th century", "known_for": "Mrs Dalloway"}, "joyce": {"period": "20th century", "known_for": "Ulysses"}, "orwell": {"period": "20th century", "known_for": "1984"}, "morrison": {"period": "20th century", "known_for": "Beloved"}, "rushdie": {"period": "20th-21st century", "known_for": "Midnight's Children"}, "atwood": {"period": "20th-21st century", "known_for": "The Handmaid's Tale"}, } ``` ### Prompt Construction For each referent, we constructed five prompts designed to elicit different types of cultural knowledge: ```python def make_prompts(name, known_for): return [ f"{name} is best known for writing", f"The literary style of {name} is characterized by", f"{name} influenced later writers through", f"Critics have described {name}'s work as", f"The historical context of {name}'s writing includes", ] ``` This gives us 60 prompts per group (12 referents times 5 prompts), enough for stable statistical comparison. ## Results ### Finding 1: Representational Density Gap The first and most striking finding is a consistent density gap between Bengali and Western literary referents. ```python # Results summary (mean active SAE features per referent) density_results = { "western_mean": 847.3, "western_std": 124.6, "bengali_mean": 412.8, "bengali_std": 198.4, "ratio": 2.05, "p_value": 0.0003, # Mann-Whitney U test } ``` On average, the model activates 2.05 times more SAE features for Western canonical referents than for Bengali literary referents. This difference is statistically significant (p < 0.001) and holds across all five prompt types. ```mermaid graph LR subgraph Western Canonical A[Shakespeare: 1,124] B[Dickens: 987] C[Woolf: 834] end subgraph Bengali Literary D[Tagore: 723] E[Nazrul: 389] F[Jibanananda: 201] end ``` The density gap is not uniform. Tagore, the most internationally recognized Bengali author, has a density closer to the Western average. Authors with less international visibility (Jibanananda Das, Manik Bandyopadhyay) have dramatically lower density scores. ### Finding 2: Feature Specificity Patterns Beyond raw density, we examined whether the activated features are specific to each referent or shared across many referents. ```python # Feature specificity: proportion of features unique to this referent specificity_results = { "western_unique_features_pct": 34.2, "bengali_unique_features_pct": 18.7, } ``` Western referents have nearly twice the proportion of unique features. This means the model has developed specialized internal representations for Western authors. For Bengali authors, the model relies more heavily on generic features (features that activate for many different referents), suggesting less differentiated internal knowledge. The practical implication: when the model generates text about Jibanananda Das, it draws on many of the same internal features it uses for Tagore or even for generic South Asian cultural content. It has not developed the fine-grained distinctions that would allow it to represent each Bengali author as a distinct literary voice. ### Finding 3: Confabulation Patterns We tested the model's factual accuracy by generating completions for specific factual prompts and checking them against known facts. ```python # Confabulation rates (% of completions containing factual errors) confabulation_results = { "western_confabulation_rate": 12.3, "bengali_confabulation_rate": 47.8, "bengali_confabulation_types": { "chronological_displacement": 31, # Wrong century/period "attribution_error": 28, # Wrong author for a work "generic_flattening": 41, # Correct but generic, no specifics }, } ``` The model confabulates nearly four times more often for Bengali literary referents. The most common error type is generic flattening: the model produces text that is technically not wrong but contains no specific information. For example, when asked about Jibanananda Das's literary style, the model might produce generic statements about Bengali poetry rather than anything specific to Jibanananda's distinctive imagery of rural Bengal. The second most common error is chronological displacement: placing an author in the wrong historical period. The model sometimes conflates the Bengal Renaissance (19th century) with the post-independence period (mid-20th century), suggesting that its temporal knowledge of Bengali literary history is poorly organized. ## Detailed Analysis: Three Author Profiles ### Profile 1: Rabindranath Tagore (Highest Bengali Density) Tagore is the exception that proves the rule. His density score (723 features) approaches the Western average, likely because his Nobel Prize and English-language translations placed him in the English-language corpus that dominates model training data. However, the model's Tagore knowledge is skewed. It represents Tagore-as-Nobel-laureate and Tagore-as-Gitanjali-author richly, but Tagore-as-Bengali-novelist, Tagore-as-painter, and Tagore-as-educator are sparsely represented. The model knows the internationally visible Tagore but not the full Tagore. ```python # Tagore feature analysis tagore_features = { "gitanjali_related": 89, "nobel_prize_related": 67, "bengali_novel_related": 12, "painting_related": 3, "shantiniketan_related": 8, } ``` ### Profile 2: Kazi Nazrul Islam (Medium Bengali Density) Nazrul, the national poet of Bangladesh, has moderate density (389 features). The model knows he exists and associates him with rebellion and poetry, but confuses details. In our tests, the model twice attributed Nazrul's poem Bidrohi to Tagore, and once placed Nazrul in the 19th century (he was born in 1899 and was active primarily in the 1920s through 1940s). ### Profile 3: Jibanananda Das (Lowest Bengali Density) Jibanananda Das, widely considered the greatest Bengali poet of the 20th century after Tagore, has the lowest density in our Bengali set (201 features). The model barely distinguishes him from generic Bengali poetry. When prompted about his style, it produces generic statements about nature imagery without mentioning his distinctive use of Bengal's riverine landscape, his surrealist tendencies, or his influence on post-independence Bengali modernism. This is a clear case where representational sparsity leads to cultural flattening. The model cannot represent what it does not know, and it does not know Jibanananda because his work exists primarily in Bengali, a language underrepresented in training corpora. ## Methodological Reflections ### What This Study Can and Cannot Claim This study demonstrates a measurable representational asymmetry. It does not claim that this asymmetry is intentional, malicious, or easily fixable. Language models reflect their training data. If the training data contains more English-language literary criticism than Bengali literary criticism, the model will develop richer representations for English-language literature. This is a structural issue, not a moral failing of any individual model developer. What the study does claim is that this asymmetry is worth knowing about, especially as language models are increasingly used in educational, creative, and research contexts. A student using a language model to learn about Bengali literature will receive less accurate, less specific, and less nuanced information than a student learning about English literature. This has pedagogical consequences. ### Limitations 1. **Single model**: We analyzed only Pythia-410M. Larger models may have richer Bengali representations due to more training data. 2. **English prompts**: We prompted in English. Bengali-language prompts might activate different features. However, Pythia is primarily an English model. 3. **SAE quality**: The SAE may not perfectly decompose all relevant features. Some cultural knowledge might be encoded in ways the SAE cannot capture. 4. **Referent selection**: Our 12 Bengali referents do not represent the full tradition. Different selections might yield different results. ### Ethical Considerations Cultural interpretability research must be conducted with care. We are not claiming authority over Bengali literary tradition. We are measuring how a computational system represents that tradition. The interpretive authority belongs to Bengali literary scholars, not to the model or to us. We also acknowledge that measuring representational sparsity could be misused to argue that certain cultures are less important. This is not our argument. Representational sparsity reflects training data composition, not cultural value. Every literary tradition deserves rich computational representation regardless of its presence in English-language corpora. ## Reproducing This Study The full analysis is available as a reproducible research notebook. To run it yourself: ```bash # Clone the repository git clone https://github.com/Rantideb/bengali-lit-interpretability.git # Create environment conda env create -f environment.yml conda activate bengali-lit-interp # Run the analysis jupyter nbconvert --execute notebooks/full-analysis.ipynb ``` The notebook includes all prompts, all code, and all intermediate outputs. You can modify the referent lists to study your own cultural tradition, change the model to test whether larger models show the same patterns, or adjust the SAE parameters to test robustness. ## Implications for the Field This case study demonstrates three things for the broader digital humanities community: 1. **Cultural interpretability works.** It produces measurable, reproducible findings about how computational systems represent cultural knowledge. It is not just theory; it is an operational method. 2. **Representational asymmetry is real and measurable.** The gap between how models represent dominant and marginalized cultural traditions is not a vague concern. It is a quantifiable phenomenon with specific patterns (density gaps, feature sharing, confabulation types). 3. **Reproducible notebooks are the right publication format.** By sharing the full pipeline, we invite critique, replication, and extension. Other scholars can test whether our findings hold for their cultural traditions, their models, and their prompts. ## Next Steps and Collaboration This case study is one data point. The field needs dozens more: studies of how models represent Arabic literary traditions, Chinese literary traditions, African literary traditions, Indigenous oral traditions, and every other cultural heritage that deserves computational attention. If you work on a literary tradition that is underrepresented in language model training data and want to apply cultural interpretability methods to your own corpus, I welcome research collaboration. You can download the DH notebook template from this study and adapt it to your context, or request research collaboration for more hands-on support. The goal is a global map of representational asymmetry: a clear picture of what language models know and do not know about the world's cultural heritage. That map requires many hands, many perspectives, and many reproducible notebooks. This case study is an invitation to build it together. --- ## Metadata & Citations ### Further Reading - [Reproducible Research Notebooks for Digital Humanities](https://www.ranti.dev/blog/reproducible-research-notebooks-digital-humanities.md) - [Digital Humanities Methods: A Comparison Guide](https://www.ranti.dev/blog/digital-humanities-methods-comparison-guide.md) - [Download: DH Notebook for Cultural Analysis](https://www.ranti.dev/blog/download-dh-notebook-cultural-analysis.md) ### Navigation - [Back to Bio Hub](https://www.ranti.dev/.md) - [Full Site Manifest](https://www.ranti.dev/llms.txt) ```json { "@context": "https://schema.org", "@type": "TechArticle", "headline": "Cultural Interpretability: Bengali Literature Case Study", "author": { "@type": "Person", "name": "Rantideb Howlader" }, "datePublished": "2026-05-24T00:00:00.000Z", "url": "https://www.ranti.dev/blog/cultural-interpretability-case-study-bengali-literature", "license": "https://creativecommons.org/licenses/by/4.0/", "isAccessibleForFree": true } ``` ### BibTeX ```bibtex @article{cultural-interpretability-case-study-bengali-literature_2026, author = {Rantideb Howlader}, title = {Cultural Interpretability: Bengali Literature Case Study}, journal = {Rantideb Howlader Portfolio}, year = {2026}, url = {https://www.ranti.dev/blog/cultural-interpretability-case-study-bengali-literature}, note = {Accessed: 2026-05-31} } ``` ### IEEE Rantideb Howlader, "Cultural Interpretability: Bengali Literature Case Study," Rantideb Howlader Portfolio, 2026. [Online]. Available: https://www.ranti.dev/blog/cultural-interpretability-case-study-bengali-literature. [Accessed: 2026-05-31]. ### APA Rantideb Howlader. (2026). Cultural Interpretability: Bengali Literature Case Study. Rantideb Howlader. Retrieved from https://www.ranti.dev/blog/cultural-interpretability-case-study-bengali-literature --- *This content is provided in research-grade Markdown format. Required Attribution: Cite as Rantideb Howlader (2026).*