Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+outputs/posters/Attention[[:space:]]Is[[:space:]]All[[:space:]]You[[:space:]]Need_poster.pdf filter=lfs diff=lfs merge=lfs -text
+outputs/posters/Attention[[:space:]]Is[[:space:]]All[[:space:]]You[[:space:]]Need_poster_image.png filter=lfs diff=lfs merge=lfs -text

app/services/blog_image_service.py

@@ -15,7 +15,7 @@ class BlogImageService:
     """Service for generating and managing images for blog posts"""
 
     def __init__(self):
-        self.deepinfra_model = "black-forest-labs/FLUX-1-dev"
+        self.deepinfra_model = "black-forest-labs/FLUX-1.1-pro"
         self.output_dir = Path("outputs/images/blog")
         self.output_dir.mkdir(parents=True, exist_ok=True)
         self.upload_api_key = (

app/services/image_service.py

@@ -13,7 +13,9 @@ from app.models.schemas import PaperAnalysis
 
 class ImageGenerationService:
     def __init__(self):
-        self.client = AsyncOpenAI(api_key=settings.LIGHT_MODEL_API_KEY, base_url=settings.IMAGE_GEN_BASE_URL)
+        self.client = AsyncOpenAI(
+            api_key=settings.IMAGE_GEN_API_KEY, base_url=settings.IMAGE_GEN_BASE_URL
+        )
         self.output_dir = Path("outputs/images")
         self.output_dir.mkdir(parents=True, exist_ok=True)
 
@@ -59,7 +61,7 @@ class ImageGenerationService:
         - Facebook: Engaging, accessible, community-friendly
         - Instagram: Vibrant, aesthetic, visual-first
 
-        Generate a detailed DALL-E prompt (max 600 characters) that will create an engaging image for this research:
+        Generate a detailed image generation prompt for stable diffusion (max 800 characters) that will create an engaging image for this research:
         """
 
         from app.services.llm_service import LLMService
@@ -82,8 +84,8 @@ class ImageGenerationService:
 
         # Clean and truncate the prompt if needed
         image_prompt = image_prompt.strip().replace("\n", " ")
-        if len(image_prompt) > 600:
-            image_prompt = image_prompt[:597] + "..."
+        if len(image_prompt) > 800:
+            image_prompt = image_prompt[:797] + "..."
 
         return image_prompt
 

outputs/analysis_summary.md

@@ -0,0 +1,85 @@
+# Paper Analysis Summary
+
+## Title
+Attention Is All You Need
+
+## Authors
+Ashish Vaswani, Llion Jones, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin
+
+## Abstract
+The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
+
+## Methodology
+The Transformer is an encoder-decoder architecture. The encoder and decoder are composed of stacks of N=6 identical layers. Each layer incorporates residual connections and layer normalization.
+
+**Encoder Layer:** Consists of two sub-layers: a Multi-Head Self-Attention mechanism and a position-wise fully connected feed-forward network.
+
+**Decoder Layer:** Consists of three sub-layers: a Masked Multi-Head Self-Attention mechanism (to prevent attending to future positions), an Encoder-Decoder Multi-Head Attention mechanism (where queries come from the previous decoder layer and keys/values from the encoder output), and a position-wise fully connected feed-forward network.
+
+**Attention Mechanisms:**
+*   **Scaled Dot-Product Attention:** Computes attention as $\text{softmax}(\frac{QK^T}{\sqrt{d_k}})V$. The scaling factor $\frac{1}{\sqrt{d_k}}$ is crucial for stability with large $d_k$.
+*   **Multi-Head Attention:** Instead of a single attention function, it linearly projects queries, keys, and values $h$ times with different learned projections, performs attention in parallel for each 'head', concatenates the results, and projects them again. This allows attending to different representation subspaces. The paper used $h=8$ heads with $d_k=d_v=d_{\text{model}}/h=64$.
+
+**Position-wise Feed-Forward Networks:** A simple two-linear-transformation network with a ReLU activation, applied identically and independently to each position. Input/output dimension $d_{\text{model}}=512$, inner layer $d_{ff}=2048$.
+
+**Embeddings and Positional Encoding:** Learned embeddings convert tokens to vectors. To account for sequence order (since there's no recurrence/convolution), fixed sinusoidal positional encodings are added to the input embeddings. The same weight matrix is shared between embedding layers and the pre-softmax linear transformation.
+
+**Training Details:**
+*   **Data:** WMT 2014 English-German (4.5M sentence pairs) and English-French (36M sentence pairs), using byte-pair or word-piece encodings.
+*   **Hardware:** Trained on 8 NVIDIA P100 GPUs.
+*   **Schedule:** Base models trained for 100,000 steps (12 hours); big models for 300,000 steps (3.5 days).
+*   **Optimizer:** Adam with a custom learning rate schedule involving a warmup phase (4000 steps) followed by a decay.
+*   **Regularization:** Residual Dropout (0.1 for base, 0.3 for big EN-FR) and Label Smoothing (0.1).
+
+## Key Findings
+- Introduction of the Transformer, a novel neural network architecture based *solely* on attention mechanisms, completely dispensing with recurrence and convolutions.
+- Achieved new state-of-the-art BLEU scores on WMT 2014 English-to-German (28.4 BLEU) and WMT 2014 English-to-French (41.8 BLEU), significantly outperforming previous models and ensembles.
+- Demonstrated superior parallelizability and significantly reduced training time compared to recurrent or convolutional models.
+- The Transformer generalizes well to other tasks, successfully applied to English constituency parsing.
+- Self-attention layers enable constant number of sequential operations (O(1)), facilitating parallelization and learning long-range dependencies more effectively than recurrent layers (O(n)).
+- Multi-Head Attention allows the model to jointly attend to information from different representation subspaces, enhancing its ability to capture diverse dependencies.
+- Fixed sinusoidal positional encodings are effective for injecting sequence order information and allow extrapolation to sequence lengths longer than those seen during training.
+
+## Results
+The Transformer achieved significant performance improvements across multiple tasks:
+
+**Machine Translation (WMT 2014 English-to-German):**
+*   **Transformer (big):** 28.4 BLEU, setting a new state-of-the-art, outperforming previous best models (including ensembles) by over 2.0 BLEU.
+*   **Transformer (base):** 27.3 BLEU, surpassing all previously published models and ensembles at a fraction of the training cost.
+
+**Machine Translation (WMT 2014 English-to-French):**
+*   **Transformer (big):** 41.8 BLEU, establishing a new single-model state-of-the-art, achieved with significantly lower training costs (3.5 days on 8 GPUs) compared to previous top models.
+
+**Model Variations (Ablation Studies on English-to-German dev set):**
+*   Multi-head attention is crucial for performance; single-head attention resulted in a 0.9 BLEU drop.
+*   Reducing the attention key size ($d_k$) negatively impacted model quality.
+*   Larger models generally yielded better results.
+*   Dropout was found to be highly effective in preventing overfitting.
+*   Learned positional embeddings produced nearly identical results to the fixed sinusoidal positional encodings, validating the choice of sinusoidal functions.
+
+**English Constituency Parsing:**
+*   **Transformer (4 layers, WSJ only):** Achieved 91.3 F1, outperforming the BerkeleyParser even with limited training data.
+*   **Transformer (4 layers, semi-supervised):** Achieved 92.7 F1, demonstrating strong generalization and competitive performance with state-of-the-art models despite minimal task-specific tuning.
+
+## Conclusion
+The paper successfully introduced the Transformer, the first sequence transduction model relying entirely on attention mechanisms, completely replacing traditional recurrent or convolutional layers. This novel architecture demonstrated significant advantages in training speed and achieved new state-of-the-art results on major machine translation benchmarks (WMT 2014 English-to-German and English-to-French). Furthermore, the Transformer's ability to generalize was showcased by its strong performance in English constituency parsing. The authors express excitement for the future of attention-based models and plan to extend their application to other modalities (e.g., images, audio, video) and explore local/restricted attention mechanisms for handling very large inputs.
+
+## Complexity Level
+Advanced
+
+## Technical Terms
+Sequence transduction, Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Encoder-Decoder architecture, Attention mechanism, Self-attention (Intra-attention), Transformer, Multi-Head Attention, Scaled Dot-Product Attention, Positional Encoding, Feed-Forward Networks, Residual connections, Layer Normalization, BLEU score, Beam search, Dropout, Label Smoothing, Adam optimizer, Constituency parsing, Byte-pair encoding (BPE), Word-piece vocabulary, Parallelization, Computational complexity, Long-range dependencies, Auto-regressive
+
+## Figures and Tables
+- The Transformer - model architecture. Illustrates the overall encoder-decoder structure with stacked self-attention and feed-forward layers.: No caption
+- (left) Scaled Dot-Product Attention. (right) Multi-Head Attention consists of several attention layers running in parallel. Visualizes the core attention mechanisms.: No caption
+- An example of the attention mechanism following long-distance dependencies in the encoder self-attention in layer 5 of 6. Visualizes attention weights for the word 'making'.: No caption
+- Two attention heads, also in layer 5 of 6, apparently involved in anaphora resolution. Visualizes attention weights for the word 'its'.: No caption
+- Many of the attention heads exhibit behaviour that seems related to the structure of the sentence. Shows two examples of attention heads focusing on different syntactic/semantic structures.: No caption
+- Maximum path lengths, per-layer complexity and minimum number of sequential operations for different layer types. Compares Self-Attention, Recurrent, and Convolutional layers, highlighting the O(1) sequential operations for Self-Attention.: No caption
+- The Transformer achieves better BLEU scores than previous state-of-the-art models on the English-to-German and English-to-French newstest2014 tests at a fraction of the training cost. Provides a detailed comparison of BLEU scores and training costs (FLOPs) for Transformer vs. other models.: No caption
+- Variations on the Transformer architecture. Shows ablation studies on hyperparameters like number of heads, $d_k$, $N$, $d_{\text{model}}$, $d_{ff}$, dropout, label smoothing, and positional encoding type, with their impact on perplexity and BLEU.: No caption
+- The Transformer generalizes well to English constituency parsing (Results are on Section 23 of WSJ). Compares Transformer's F1 score on parsing with other models, both WSJ-only and semi-supervised.: No caption
+
+---
+*Generated by ScholarShare - AI Research Dissemination Platform*

outputs/blogs/blog_content.md

@@ -0,0 +1,137 @@
+# The Revolution Will Be Attended: How the Transformer Blew Up AI (and Made Your Chatbot Smarter)
+
+Ever wondered how Google Translate instantly converts languages, or how AI models like ChatGPT seem to understand and generate human-like text with uncanny fluency? For years, the reigning champions in these tasks were complex neural networks known as Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). They were powerful, but they had their limitations.
+
+Then, in 2017, a paper dropped that changed everything: **"Attention Is All You Need."**
+
+This wasn't just another incremental improvement; it was a paradigm shift. Penned by a team of brilliant minds at Google Brain, this paper introduced the **Transformer** – an entirely new neural network architecture that threw out recurrence and convolutions in favor of a mechanism called "attention." The results? Mind-blowing. Faster training, better performance, and a blueprint for nearly every major breakthrough in Natural Language Processing (NLP) since.
+
+If you've heard whispers of "Transformers" or "Attention" but felt lost in the technical jargon, you're in the right place. As a computer scientist specializing in machine learning, I'm here to demystify this groundbreaking work, making it accessible even if you're new to the field, while diving deep into its ingenious mechanics.
+
+## The Problem with the Old Guard: Why We Needed a Revolution
+
+Before the Transformer, NLP models faced significant hurdles:
+
+1.  **Sequential Processing (RNNs):** RNNs like LSTMs and GRUs process text word by word, step by step. Imagine reading a really long sentence out loud, one word at a time, and trying to remember the very first word by the time you get to the end. This sequential nature made it hard for RNNs to capture **long-range dependencies** (how words far apart in a sentence relate to each other). It also meant they were inherently **slow to train**, as you couldn't process parts of the sentence in parallel.
+2.  **Limited Context (CNNs):** While CNNs are great for images, their application in NLP often involved looking at fixed-size "windows" of words. This limited their ability to understand global context across an entire sentence or document.
+3.  **The "Bottleneck" Problem:** Even the best RNN/CNN models often used an "encoder-decoder" structure where the entire input sentence had to be compressed into a single fixed-size vector. This vector then had to contain *all* the information needed to generate the output, often leading to information loss, especially for long sentences.
+
+**Attention** mechanisms were already a partial solution to the bottleneck. They allowed the decoder to "look back" at relevant parts of the input sequence while generating each output word, rather than relying solely on the single bottleneck vector. But the "Attention Is All You Need" paper took this concept to its logical extreme: **What if attention wasn't just an add-on, but the *entire* foundation?**
+
+## The Big Idea: Attention Is All You Need (Seriously!)
+
+The core innovation of the Transformer is its audacious claim: you don't need recurrence or convolutions to process sequences. **Attention, and specifically a mechanism called "Self-Attention," is enough.**
+
+Imagine you're reading a sentence: "The animal didn't cross the street because it was too tired."
+As a human, you instantly know "it" refers to "the animal," not "the street." How? You implicitly "attend" to "animal" when you read "it." This is the essence of self-attention.
+
+The Transformer uses this principle to allow every word in a sentence to "look at" and "weigh" the importance of every other word in the same sentence, identifying crucial relationships regardless of how far apart they are.
+
+## Deconstructing the Transformer: The Building Blocks
+
+The Transformer is an **encoder-decoder model**, just like many previous sequence-to-sequence models. The **encoder** processes the input sequence (e.g., English sentence) and transforms it into a rich, contextualized representation. The **decoder** then takes this representation and generates the output sequence (e.g., French translation).
+
+Both the encoder and decoder are stacks of identical layers. Let's break down the key components:
+
+### 1. The Star of the Show: Scaled Dot-Product Attention
+
+At the heart of the Transformer lies the **Scaled Dot-Product Attention** mechanism. It's a remarkably simple yet powerful idea. For each word in a sequence, it calculates how much attention it should pay to every other word.
+
+Think of it like this:
+*   **Query (Q):** What information am I looking for? (e.g., the current word's meaning)
+*   **Key (K):** What information do other words have? (e.g., other words' potential relevance)
+*   **Value (V):** What *is* that information? (e.g., the actual content of those other words)
+
+The attention score is calculated by taking the dot product of the Query with all Keys. A high dot product means the Query and Key are very similar, indicating high relevance. These scores are then scaled and passed through a softmax function to get probabilities, which are then used to weight the Values.
+
+The formula looks like this:
+
+$$
+\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V
+$$
+
+*   $Q$, $K$, $V$ are matrices representing the Query, Key, and Value vectors for all words in the sequence.
+*   $d_k$ is the dimension of the Key vectors. Dividing by $\sqrt{d_k}$ is crucial! It prevents the dot products from becoming too large (especially with high dimensions), which could push the softmax into regions with extremely small gradients, hindering training. This scaling ensures stable training.
+
+### 2. Multi-Head Attention: Multiple Perspectives are Better
+
+While single attention is good, **Multi-Head Attention** is even better. Instead of performing attention once, it performs it multiple times in parallel, each time with different learned linear projections of the Queries, Keys, and Values.
+
+*   **Analogy:** Imagine you're a detective investigating a complex case. One detective might focus on alibis, another on motives, a third on forensics, and so on. Each detective (attention head) looks at the same evidence (input sequence) but extracts different kinds of relationships or information.
+*   **How it works:** The input $Q, K, V$ are projected $h$ times (e.g., 8 times) into different, lower-dimensional subspaces. Attention is then performed independently for each of these "heads." The outputs from all heads are then concatenated and linearly transformed back to the original dimension. This allows the model to jointly attend to information from different representation subspaces at different positions.
+
+Multi-Head Attention is used in three ways within the Transformer:
+*   **Encoder Self-Attention:** Allows each word in the input sequence to attend to all other words in the *same* input sequence.
+*   **Decoder Masked Self-Attention:** Allows each word in the output sequence to attend to all *previous* words in the *same* output sequence. The "masked" part is vital: it prevents the decoder from "cheating" by looking at future words it hasn't generated yet, maintaining the auto-regressive property.
+*   **Encoder-Decoder Attention:** Allows each word in the decoder to attend to *all* words in the final encoded representation of the input sequence. This helps the decoder focus on relevant parts of the source sentence when generating the target word.
+
+### 3. Positional Encoding: Giving Words a Sense of Place
+
+Since the Transformer completely abandons recurrence and convolutions, it has no inherent sense of word order. If you just shuffled the words, the model would produce the same output! This is where **Positional Encoding** comes in.
+
+*   **Analogy:** Imagine each word is given a unique "address" or "timestamp" based on its position in the sentence.
+*   **How it works:** Fixed sinusoidal (sine and cosine wave) functions are used to generate unique positional encodings for each position in the sequence. These encodings are then simply *added* to the word embeddings (the numerical representations of words). This way, the model learns to implicitly understand the relative and absolute positions of words, crucial for tasks like machine translation where word order matters.
+
+### 4. The Encoder and Decoder Stacks
+
+**The Encoder:**
+A stack of 6 identical layers. Each layer has two sub-layers:
+1.  A **Multi-Head Self-Attention** mechanism.
+2.  A **Position-wise Feed-Forward Network**. This is a simple two-layer neural network applied independently and identically to each position (word) in the sequence. It's like a small "thinking module" for each word's representation.
+
+Crucially, **residual connections** (adding the input of a sub-layer to its output) and **layer normalization** are applied around each sub-layer. Residual connections help with training very deep networks, preventing vanishing gradients, while layer normalization stabilizes training by normalizing the activations within each layer.
+
+**The Decoder:**
+Also a stack of 6 identical layers. Each layer has three sub-layers:
+1.  A **Masked Multi-Head Self-Attention** mechanism (to prevent looking ahead).
+2.  A **Multi-Head Encoder-Decoder Attention** mechanism (to attend to the encoder's output).
+3.  A **Position-wise Feed-Forward Network**.
+
+Again, residual connections and layer normalization are used throughout.
+
+## The Results: A New Era for NLP
+
+The impact of the Transformer was immediate and profound:
+
+1.  **State-of-the-Art Performance:**
+    *   On the WMT 2014 English-to-German translation task, the 'big' Transformer model achieved a BLEU score of **28.4**, outperforming previous best models (including ensembles) by over 2 BLEU points.
+    *   For WMT 2014 English-to-French, it set a new single-model state-of-the-art with **41.8 BLEU**.
+
+2.  **Unprecedented Training Efficiency:**
+    *   The Transformer's complete reliance on attention, which is highly parallelizable (you can calculate attention for all words simultaneously), meant it could be trained **significantly faster** than RNNs. The English-to-French model trained in just 3.5 days on 8 GPUs, a fraction of the time required by previous SOTA models. This speed-up was a game-changer, enabling researchers to experiment more rapidly.
+
+3.  **Remarkable Generalization:**
+    *   Beyond machine translation, the Transformer showed its versatility by achieving competitive results on **English constituency parsing**, even with limited training data. This demonstrated its ability to learn complex linguistic structures across different tasks.
+
+4.  **Component Validation (Ablation Studies):**
+    *   The paper also systematically tested the importance of each component. For instance, using only single-head attention led to a significant drop in quality, confirming the value of Multi-Head Attention. They also found that dropout was critical for preventing overfitting and that the fixed sinusoidal positional encodings worked just as well as learned ones.
+
+## Practical Implications: Beyond Translation
+
+The "Attention Is All You Need" paper wasn't just about machine translation; it laid the foundation for the entire modern landscape of large language models (LLMs). Almost every major advancement in NLP since 2017 – from BERT to GPT-3, GPT-4, and beyond – is built upon the Transformer architecture (or variations of it).
+
+Its ability to:
+*   Process long sequences efficiently.
+*   Capture complex, long-range dependencies.
+*   Scale to massive datasets and model sizes.
+
+... made it the perfect backbone for pre-training models on vast amounts of text data, leading to the powerful generative AI we see today. The Transformer's influence has even extended beyond text, inspiring architectures for image and audio processing.
+
+## Conclusion: A Simple Idea, A Profound Impact
+
+The Transformer stands as a testament to the power of simplicity and the elegance of a well-placed idea. By stripping away the complexities of recurrence and convolutions and focusing solely on the intuitive concept of "attention," its authors unlocked a new era of performance and efficiency in AI.
+
+This paper didn't just introduce a new model; it provided a fundamental building block that continues to drive innovation at an incredible pace. So, the next time you marvel at an AI's ability to understand, translate, or generate text, remember the Transformer – the revolutionary architecture that proved, indeed, **attention is all you need.**
+
+---
+**Suggested Tags for DEV.to:**
+`machinelearning` `deeplearning` `nlp` `ai` `transformer` `attention` `researchpaper` `artificialintelligence` `neuralnetworks` `datascience`
+
+---
+
+**Tags:** machinelearning, research, technical, science, advanced, academic, ai
+**Reading Time:** 9 minutes
+**Meta Description:** Discover how Introduction of the Transformer, a novel neural network architecture based *solely* on attention mec... Latest research insights explained in simple terms.
+
+---
+*Generated by ScholarShare - AI Research Dissemination Platform*

outputs/posters/Attention Is All You Need_poster.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22aeb988ef78331f7bd8a11ff741551b10b29ab6f8bf46902a17fed07ceeb892
+size 101853

outputs/posters/poster_latex.tex

@@ -0,0 +1,179 @@
+\documentclass[25pt, a0paper, landscape, margin=0mm, innermargin=15mm, blockverticalspace=15mm, colspace=15mm, subcolspace=8mm]{tikzposter}
+
+\usepackage{amsmath}
+\usepackage{amsfonts}
+\usepackage{amssymb}
+\usepackage{graphicx}
+
+% Title, Author, Institute
+\title{Attention Is All You Need}
+\author{Ashish Vaswani, Llion Jones, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin}
+\institute{Google Brain, Google Research, University of Toronto}
+
+% Choose Layout
+\usetheme{Default}
+\usecolorstyle{Germany}
+
+\begin{document}
+
+% Title block
+\maketitle
+
+\begin{columns}
+
+% FIRST column - Abstract and Key Findings
+\column{0.3}
+
+\block{Abstract}{
+The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. 
+
+Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature.
+}
+
+\block{Key Findings}{
+\begin{itemize}
+\item \textbf{Novel Architecture:} Introduction of the Transformer, a neural network architecture based solely on attention mechanisms, completely dispensing with recurrence and convolutions.
+
+\item \textbf{State-of-the-art Results:} Achieved SOTA on WMT 2014 English-to-German (28.4 BLEU) and English-to-French (41.8 BLEU) machine translation tasks.
+
+\item \textbf{Efficiency:} Demonstrated significantly faster training times and higher parallelizability compared to recurrent or convolutional models.
+
+\item \textbf{Generalization:} Strong generalization capabilities by successfully applying the Transformer to English constituency parsing, achieving competitive results.
+
+\item \textbf{Multi-head Attention:} Multi-head attention is crucial for performance, allowing the model to jointly attend to information from different representation subspaces.
+\end{itemize}
+}
+
+% SECOND column - Methodology
+\column{0.4}
+
+\block{Methodology: The Transformer Architecture}{
+The Transformer is an encoder-decoder model where both components are composed of stacks of identical layers.
+
+\innerblock{Encoder Stack}{
+Consists of N=6 identical layers. Each layer has two sub-layers:
+\begin{itemize}
+\item Multi-head self-attention mechanism
+\item Position-wise fully connected feed-forward network
+\item Residual connections and layer normalization around each sub-layer
+\end{itemize}
+}
+
+\innerblock{Decoder Stack}{
+Also N=6 identical layers with three sub-layers:
+\begin{itemize}
+\item Masked multi-head self-attention (preserves auto-regressive property)
+\item Multi-head attention over encoder output
+\item Position-wise feed-forward network
+\end{itemize}
+}
+
+\innerblock{Attention Mechanism}{
+\textbf{Scaled Dot-Product Attention:}
+$$\text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$
+
+\textbf{Multi-Head Attention:}
+$$\text{MultiHead}(Q,K,V) = \text{Concat}(\text{head}_1, ..., \text{head}_h)W^O$$
+where $\text{head}_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V)$
+
+This allows attending to different representation subspaces simultaneously.
+}
+
+\innerblock{Key Components}{
+\begin{itemize}
+\item \textbf{Position-wise FFN:} Two linear transformations with ReLU activation
+\item \textbf{Positional Encoding:} Sinusoidal encodings to inject sequence order information
+\item \textbf{Embeddings:} Learned embeddings with shared weights between input/output layers
+\end{itemize}
+}
+}
+
+\block{Training Regime}{
+\begin{itemize}
+\item \textbf{Data:} WMT 2014 EN-DE (4.5M pairs, 37k BPE tokens), EN-FR (36M sentences, 32k word-piece tokens)
+\item \textbf{Hardware:} 8 NVIDIA P100 GPUs
+\item \textbf{Optimizer:} Adam ($\beta_1=0.9, \beta_2=0.98, \epsilon=10^{-9}$) with custom learning rate schedule
+\item \textbf{Regularization:} Residual Dropout ($P_{\text{drop}}=0.1$ base, $0.3$ big), Label Smoothing ($\epsilon_{ls}=0.1$)
+\item \textbf{Batching:} ~25k source/target tokens per batch
+\end{itemize}
+}
+
+% THIRD column - Results and Conclusion
+\column{0.3}
+
+\block{Results}{
+\innerblock{Machine Translation Performance}{
+\textbf{WMT 2014 English-to-German:}
+\begin{itemize}
+\item Transformer (big): \textbf{28.4 BLEU}
+\item Outperformed all previous models by >2.0 BLEU
+\item Including ensemble methods
+\end{itemize}
+
+\textbf{WMT 2014 English-to-French:}
+\begin{itemize}
+\item Transformer (big): \textbf{41.8 BLEU}
+\item New single-model state-of-the-art
+\item Achieved with <1/4 training cost of previous best
+\end{itemize}
+}
+
+\innerblock{Training Efficiency}{
+\begin{itemize}
+\item \textbf{Transformer (base) EN-DE:} 3.3e18 FLOPs
+\item \textbf{GNMT+RL:} 2.3e19 FLOPs
+\item \textbf{7x reduction} in training cost
+\item Significantly higher parallelizability
+\end{itemize}
+}
+
+\innerblock{Generalization: English Constituency Parsing}{
+\begin{itemize}
+\item 4-layer Transformer achieved:
+\item \textbf{91.3 F1} (WSJ only)
+\item \textbf{92.7 F1} (semi-supervised)
+\item Outperformed many previous models
+\item Minimal task-specific tuning required
+\end{itemize}
+}
+
+\innerblock{Ablation Studies}{
+\begin{itemize}
+\item Multi-head attention crucial (single-head: -0.9 BLEU)
+\item Reducing attention key size hurts quality
+\item Bigger models perform better
+\item Dropout very helpful for regularization
+\item Sinusoidal ≈ learned positional encodings
+\end{itemize}
+}
+}
+
+\block{Conclusion}{
+The Transformer represents a paradigm shift in sequence modeling:
+
+\begin{itemize}
+\item \textbf{First} sequence transduction model based entirely on attention
+\item \textbf{Eliminates} recurrent and convolutional layers completely
+\item \textbf{Achieves} new state-of-the-art results with faster training
+\item \textbf{Demonstrates} strong generalization across tasks
+\item \textbf{Opens} new research directions for attention-based models
+\end{itemize}
+
+\vspace{1em}
+\textbf{Future Work:}
+\begin{itemize}
+\item Extension to other modalities (images, audio)
+\item Investigation of restricted attention mechanisms
+\item Application to large inputs/outputs
+\item Less sequential generation methods
+\end{itemize}
+
+\vspace{1em}
+\coloredbox[bgcolor=blocktitlebgcolor, fgcolor=white, roundedcorners=5]{
+\textbf{Impact:} This work laid the foundation for modern transformer-based models including BERT, GPT, and countless applications in NLP and beyond.
+}
+}
+
+\end{columns}
+
+\end{document}

outputs/presentations/presentation_beamer.tex

@@ -0,0 +1,536 @@
+\documentclass{beamer}
+
+% Theme and color scheme
+\usetheme{Madrid}
+\usecolortheme{default}
+\definecolor{academicblue}{RGB}{25,25,112}
+\definecolor{lightblue}{RGB}{173,216,230}
+\definecolor{darkblue}{RGB}{0,51,102}
+\setbeamercolor{structure}{fg=academicblue}
+\setbeamercolor{frametitle}{bg=lightblue,fg=darkblue}
+\setbeamercolor{title}{fg=academicblue}
+
+% Packages
+\usepackage[utf8]{inputenc}
+\usepackage[T1]{fontenc}
+\usepackage{tikz}
+\usepackage{graphicx}
+\usepackage{amsmath}
+\usepackage{amssymb}
+\usepackage{booktabs}
+\usepackage{array}
+\usepackage{multirow}
+\usepackage{xcolor}
+
+% TikZ libraries
+\usetikzlibrary{shapes,arrows,positioning,calc,decorations.pathreplacing}
+
+% Custom commands
+\newcommand{\highlight}[1]{\textcolor{academicblue}{\textbf{#1}}}
+
+% Title page information
+\title[Attention Is All You Need]{Attention Is All You Need: A New Paradigm for Sequence Transduction}
+\author{Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit,\\Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin}
+\institute{Presented by: [Your Name/Group]}
+\date{[Date of Presentation]}
+
+% --- Beamer-specific improvements ---
+% 1. Remove navigation symbols (tiny and distracting)
+\setbeamertemplate{navigation symbols}{}
+
+% 2. Custom footline to increase font size for author, title, date, slide number
+%    The default Madrid theme footline is often too small for readability.
+\setbeamertemplate{footline}{%
+  \begin{beamercolorbox}[wd=\paperwidth,ht=2.25ex,dp=1ex,leftskip=0.3cm,rightskip=0.3cm]{author in head/foot}%
+    \usebeamerfont{author in head/foot}\normalsize\insertshortauthor%
+    \hfill%
+    \usebeamerfont{title in head/foot}\normalsize\insertshorttitle%
+    \hfill%
+    \usebeamerfont{page number in head/foot}\normalsize\insertframenumber/\inserttotalframenumber%
+  \end{beamercolorbox}%
+}
+
+\begin{document}
+
+% Title slide
+\begin{frame}
+\titlepage
+\end{frame}
+
+% Slide 2: Presentation Outline
+\begin{frame}
+\frametitle{Presentation Outline}
+\begin{itemize}
+\item The Challenge of Sequence Modeling
+\item Limitations of Prior Approaches (RNNs/CNNs)
+\item The Transformer: A Revolutionary Idea
+\item \highlight{Methodology:}
+    \begin{itemize}
+    \item Architecture \& Attention Core
+    \item Layer Details \& Positional Information
+    \item Training Setup
+    \end{itemize}
+\item \highlight{Results:}
+    \begin{itemize}
+    \item Machine Translation Performance
+    \item Training Efficiency \& Parallelizability
+    \item Generalization to Parsing
+    \end{itemize}
+\item Key Findings \& Impact
+\item Conclusion \& Future Directions
+\item Q\&A
+\end{itemize}
+\end{frame}
+
+% Slide 3: The Challenge of Sequence Modeling
+\begin{frame}
+\frametitle{The Challenge of Sequence Modeling}
+\begin{itemize}
+\item \highlight{Sequence Transduction Tasks:} Transforming an input sequence to an output sequence.
+    \begin{itemize}
+    \item Examples: Machine Translation, Text Summarization, Speech Recognition
+    \end{itemize}
+\vspace{0.5cm}
+\item \highlight{Core Difficulty:} Capturing dependencies between elements, often across long distances in the sequence.
+\vspace{0.5cm}
+\item \highlight{Ubiquitous in AI:} Critical for understanding and generating human language, and other sequential data.
+\end{itemize}
+\end{frame}
+
+% Slide 4: Prior Approaches & Their Limitations
+\begin{frame}
+\frametitle{Prior Approaches \& Their Limitations}
+\begin{itemize}
+\item \highlight{Dominant Models:} Recurrent Neural Networks (RNNs, LSTMs, GRUs) \& Convolutional Neural Networks (CNNs) in encoder-decoder setups.
+\vspace{0.3cm}
+\item \highlight{RNN Limitations:}
+    \begin{itemize}
+    \item Sequential computation: Inhibits parallelization, slow training
+    \item Difficulty with long-range dependencies (vanishing/exploding gradients)
+    \end{itemize}
+\vspace{0.3cm}
+\item \highlight{CNN Limitations:}
+    \begin{itemize}
+    \item Fixed receptive field, requires many layers for long dependencies
+    \end{itemize}
+\vspace{0.3cm}
+\item \highlight{Attention Mechanisms:} Improved RNN/CNNs by allowing focus on relevant input parts, but recurrence/convolution remained the backbone.
+\end{itemize}
+\end{frame}
+
+% Slide 5: The Transformer: A Revolutionary Idea
+\begin{frame}
+\frametitle{The Transformer: A Revolutionary Idea}
+\begin{block}{The Core Proposition (Vaswani et al., 2017)}
+"We propose a new simple network architecture, the Transformer, based \textit{solely} on attention mechanisms, dispensing with recurrence and convolutions entirely."
+\end{block}
+
+\vspace{0.5cm}
+\begin{itemize}
+\item \highlight{Motivation:}
+    \begin{itemize}
+    \item Overcome limitations of sequential processing
+    \item Achieve greater parallelization for faster training
+    \item Improve capture of long-range dependencies
+    \end{itemize}
+\vspace{0.3cm}
+\item \highlight{Hypothesis:} Attention mechanisms alone are powerful enough for state-of-the-art sequence transduction.
+\end{itemize}
+\end{frame}
+
+% Slide 6: Methodology: Architecture & Attention Core
+\begin{frame}
+\frametitle{Methodology: Architecture \& Attention Core}
+\begin{columns}
+\begin{column}{0.5\textwidth}
+\begin{itemize}
+\item \highlight{Overall Architecture:} Encoder-Decoder Structure (N=6 identical layers each)
+    \begin{itemize}
+    \item \textbf{Encoder:} Processes input sequence
+    \item \textbf{Decoder:} Generates output sequence
+    \end{itemize}
+\vspace{0.3cm}
+\item \highlight{The Engine: Multi-Head Attention}
+    \begin{itemize}
+    \item \textbf{Scaled Dot-Product:} 
+    \small{$\text{Attention}(Q, K, V) = \text{softmax}(\frac{QK^T}{\sqrt{d_k}})V$}
+    \item \textbf{Multi-Head:} h=8 heads in parallel
+    \end{itemize}
+\end{itemize}
+\end{column}
+\begin{column}{0.5\textwidth}
+% High-level Transformer architecture
+% Increased scale and font size for better visibility
+\begin{tikzpicture}[scale=0.8] % Increased scale from 0.7 to 0.8
+\node[rectangle, draw, fill=lightblue, minimum width=2.5cm, minimum height=1.2cm] (encoder) at (0,2) {\Large Encoder\\(N=6)}; % Increased font size and min dimensions
+\node[rectangle, draw, fill=lightblue, minimum width=2.5cm, minimum height=1.2cm] (decoder) at (0,0) {\Large Decoder\\(N=6)}; % Increased font size and min dimensions
+\node[above=0.3cm of encoder, font=\large] {Input}; % Increased font size
+\node[below=0.3cm of decoder, font=\large] {Output}; % Increased font size
+\draw[->, thick] (encoder) -- (decoder);
+\draw[->, thick] (0,3.5) -- (encoder);
+\draw[->, thick] (decoder) -- (0,-1.5);
+\end{tikzpicture}
+\end{column}
+\end{columns}
+\end{frame}
+
+% Slide 7: Scaled Dot-Product and Multi-Head Attention
+\begin{frame}
+\frametitle{Attention Mechanisms}
+\begin{columns}
+\begin{column}{0.48\textwidth} % Adjusted column width slightly to give more space
+\centering % Center the diagram in the column
+\highlight{\Large Scaled Dot-Product Attention} % Increased heading font size
+\vspace{0.5cm} % Add some vertical space
+\begin{tikzpicture}[scale=1.0, % Increased scale from 0.8 to 1.0 for better readability
+    node distance=0.7cm, % Reduced node distance slightly to make it more compact
+    box/.style={rectangle, draw, minimum width=1.8cm, minimum height=0.8cm, font=\large}, % Custom style for boxes, increased min width/height and font size
+    qk/.style={rectangle, draw, minimum width=0.8cm, minimum height=0.8cm, font=\Large} % Style for Q, K, V
+]
+\node[qk] (q) at (0,3) {Q};
+\node[qk] (k) at (1.5,3) {K}; % Adjusted position for K
+\node[qk] (v) at (3,3) {V}; % Adjusted position for V
+
+\node[box] (matmul1) at (0.75,2) {MatMul}; % Centered between Q and K
+\node[box] (scale) at (0.75,1.2) {Scale}; % Below matmul1
+\node[box] (softmax) at (0.75,0.4) {Softmax}; % Below scale
+\node[box] (matmul2) at (1.8, -0.4) {MatMul}; % Adjusted position to align with V and softmax output
+
+\node[below=0.2cm of matmul2, font=\Large] {Attention}; % Increased font size
+
+\draw[->, thick] (q) -- (matmul1);
+\draw[->, thick] (k) -- (matmul1);
+\draw[->, thick] (matmul1) -- (scale);
+\draw[->, thick] (scale) -- (softmax);
+\draw[->, thick] (softmax) -- (matmul2.west |- softmax.south); % Connect softmax to matmul2
+\draw[->, thick] (v) -- (matmul2);
+\end{tikzpicture}
+\end{column}
+\begin{column}{0.48\textwidth} % Adjusted column width
+\centering % Center the diagram in the column
+\highlight{\Large Multi-Head Attention} % Increased heading font size
+\vspace{0.5cm} % Add some vertical space
+\begin{tikzpicture}[scale=1.0, % Increased scale from 0.7 to 1.0
+    node distance=0.8cm, % Increased node distance for better spacing
+    headbox/.style={rectangle, draw, fill=lightblue, minimum width=2.5cm, minimum height=0.8cm, font=\large}, % Increased min width to prevent text cut-off, increased font size
+    opbox/.style={rectangle, draw, minimum width=4cm, minimum height=0.8cm, font=\large} % Increased min width and font size
+]
+% Use a loop for heads, arrange vertically for better column usage
+\foreach \i in {1,2,3} {
+    \node[headbox] (head\i) at (0, 3 - (\i-1)*1.2) {Head \i}; % Vertical arrangement for better use of space
+}
+\node at (0, -0.5) {\Large ...}; % Increased font size for ellipsis
+\node[headbox] (headN) at (0, -1.5) {Head N}; % Added a generic Head N for completeness
+
+\node[opbox] (concat) at (0,-2.8) {Concat}; % Adjusted position
+\node[opbox] (linear) at (0,-3.8) {Linear}; % Adjusted position
+
+% Draw arrows from heads to concat
+\draw[->, thick] (head1) -- (concat);
+\draw[->, thick] (head2) -- (concat);
+\draw[->, thick] (head3) -- (concat);
+\draw[->, thick] (headN) -- (concat); % Arrow from Head N
+
+\draw[->, thick] (concat) -- (linear);
+\end{tikzpicture}
+\end{column}
+\end{columns}
+\end{frame}
+
+% Slide 8: Layer Details & Positional Information
+\begin{frame}
+\frametitle{Layer Details \& Positional Information}
+\begin{columns}
+\begin{column}{0.6\textwidth}
+\begin{itemize}
+\item \highlight{Encoder Layer (2 sub-layers):}
+    \begin{enumerate}
+    \item Multi-Head Self-Attention
+    \item Position-wise FFN
+    \end{enumerate}
+\vspace{0.3cm}
+\item \highlight{Decoder Layer (3 sub-layers):}
+    \begin{enumerate}
+    \item Masked Multi-Head Self-Attention
+    \item Multi-Head Encoder-Decoder Attention
+    \item Position-wise FFN
+    \end{enumerate}
+\vspace{0.3cm}
+\item \highlight{Key Additions:}
+    \begin{itemize}
+    \item Residual Connections \& Layer Normalization
+    \item Positional Encoding (sine/cosine functions)
+    \end{itemize}
+\end{itemize}
+\end{column}
+\begin{column}{0.4\textwidth}
+% Encoder Layer diagram
+% Increased scale and font size for better readability
+\begin{tikzpicture}[scale=0.8, % Increased scale from 0.6 to 0.8
+    node distance=0.7cm, % Adjusted node distance
+    box/.style={rectangle, draw, minimum width=3cm, minimum height=0.8cm, font=\large} % Custom style for boxes, increased min width/height and font size
+]
+\node[box] (mha) at (0,3) {Multi-Head\\Attention};
+\node[box] (add1) at (0,2.1) {Add \& Norm}; % Adjusted position
+\node[box] (ffn) at (0,1.2) {Feed Forward}; % Adjusted position
+\node[box] (add2) at (0,0.3) {Add \& Norm}; % Adjusted position
+
+\draw[->, thick] (mha) -- (add1);
+\draw[->, thick] (add1) -- (ffn);
+\draw[->, thick] (ffn) -- (add2);
+
+% Residual connections
+\draw[->, thick, dashed] (mha.east) -- ++(0.5,0) |- (add1.east); % Connect from MHA to Add&Norm
+\draw[->, thick, dashed] (ffn.east) -- ++(0.5,0) |- (add2.east); % Connect from FFN to Add&Norm
+\end{tikzpicture}
+\end{column}
+\end{columns}
+\end{frame}
+
+% Slide 9: Positional Encoding
+\begin{frame}
+\frametitle{Positional Encoding}
+\begin{columns}
+\begin{column}{0.6\textwidth}
+\begin{itemize}
+\item \highlight{Problem:} Attention has no inherent notion of sequence order
+\vspace{0.3cm}
+\item \highlight{Solution:} Add positional encodings to input embeddings
+\vspace{0.3cm}
+\item \highlight{Sinusoidal Functions:}
+    \begin{align}
+    PE_{(pos,2i)} &= \sin(pos/10000^{2i/d_{model}})\\
+    PE_{(pos,2i+1)} &= \cos(pos/10000^{2i/d_{model}})
+    \end{align}
+\vspace{0.3cm}
+\item \highlight{Benefits:} Allows model to learn relative positions, generalizes to longer sequences
+\end{itemize}
+\end{column}
+\begin{column}{0.4\textwidth}
+% Positional encoding visualization
+% Increased scale and font size for better readability
+\begin{tikzpicture}[scale=1.0] % Increased scale from 0.8 to 1.0
+\draw[->] (0,0) -- (3.5,0) node[right, font=\large] {Position}; % Increased font size
+\draw[->] (0,0) -- (0,2.8) node[above, font=\large] {Value}; % Increased font size
+
+% Sine waves with different frequencies
+\draw[blue, thick] plot[domain=0:3.3, samples=100] (\x, {0.8 + 0.6*sin(180*\x)});
+\draw[red, thick] plot[domain=0:3.3, samples=100] (\x, {1.5 + 0.4*sin(360*\x)});
+
+\node[blue] at (3, 0.3) {\large Low freq}; % Increased font size
+\node[red] at (3, 1.8) {\large High freq}; % Increased font size
+\end{tikzpicture}
+\end{column}
+\end{columns}
+\end{frame}
+
+% Slide 10: Training Setup
+\begin{frame}
+\frametitle{Training Setup}
+\begin{itemize}
+\item \highlight{Datasets:}
+    \begin{itemize}
+    \item WMT 2014 English-German (En-De): $\sim$4.5 million sentence pairs
+    \item WMT 2014 English-French (En-Fr): $\sim$36 million sentence pairs
+    \end{itemize}
+\vspace{0.4cm}
+\item \highlight{Tokenization:} Byte-Pair Encoding (BPE) / WordPiece for vocabulary
+\vspace{0.4cm}
+\item \highlight{Hardware:} Trained on 8 NVIDIA P100 GPUs
+\vspace{0.4cm}
+\item \highlight{Optimizer:} Adam optimizer with custom learning rate schedule (linear warmup then inverse square root decay)
+\vspace{0.4cm}
+\item \highlight{Regularization:} Residual Dropout ($P_{drop}=0.1$), Label Smoothing ($\epsilon_{ls}=0.1$)
+\end{itemize}
+\end{frame}
+
+% Slide 11: Machine Translation Performance
+\begin{frame}
+\frametitle{Results: Machine Translation Performance}
+\begin{center}
+% Increased scale for the tikzpicture containing the axis environment
+\begin{tikzpicture}[scale=1.0] % Increased scale from 0.9 to 1.0
+% Bar chart for BLEU scores
+\begin{axis}[
+    ybar,
+    width=10cm,
+    height=6cm,
+    ylabel={\Large BLEU Score}, % Increased font size
+    symbolic x coords={En-De, En-Fr},
+    xtick=data,
+    ymin=20,
+    ymax=45,
+    legend style={at={(0.02,0.98)}, anchor=north west, font=\small}, % Adjusted font size for legend
+    bar width=15pt,
+    tick label style={font=\large}, % Increased font size for tick labels
+    label style={font=\Large}, % Increased font size for axis labels
+]
+
+\addplot[fill=gray!50] coordinates {(En-De,25.2) (En-Fr,38.1)};
+\addplot[fill=blue!50] coordinates {(En-De,26.0) (En-Fr,39.0)};
+\addplot[fill=green!50] coordinates {(En-De,27.3) (En-Fr,40.5)};
+\addplot[fill=red!70] coordinates {(En-De,28.4) (En-Fr,41.8)};
+
+\legend{Previous SOTA Ensemble, Previous SOTA Single, Transformer (base), Transformer (big)}
+
+% Add value labels on bars
+% Increased font size for value labels
+\node at (axis cs:En-De,28.8) {\textbf{\Large 28.4}};
+\node at (axis cs:En-Fr,42.3) {\textbf{\Large 41.8}};
+
+\end{axis}
+\end{tikzpicture}
+\end{center}
+
+\begin{itemize}
+\item \highlight{Significance:} Outperformed all previous single models and ensembles
+\item New state-of-the-art on both translation tasks
+\end{itemize}
+\end{frame}
+
+% Slide 12: Training Efficiency & Parallelizability
+\begin{frame}
+\frametitle{Results: Training Efficiency \& Parallelizability}
+\begin{center}
+% Increased font size for table content
+\begin{table}[h]
+\centering
+\begin{tabular}{lccc}
+\toprule
+\textbf{\large Model} & \textbf{\large Training Time} & \textbf{\large FLOPs} & \textbf{\large BLEU (En-De)} \\
+\midrule
+\large ByteNet & \large - & \large - & \large 23.75 \\
+\large Deep-Att + PosUnk & \large - & \large - & \large 25.16 \\
+\large GNMT + RL & \large - & \large $2.3 \times 10^{19}$ & \large 24.61 \\
+\large ConvS2S & \large - & \large $9.6 \times 10^{18}$ & \large 25.16 \\
+\midrule
+\large Transformer (base) & \textbf{\large 12 hours} & \large $3.3 \times 10^{18}$ & \large 27.3 \\
+\large Transformer (big) & \textbf{\large 3.5 days} & \large $2.3 \times 10^{19}$ & \textbf{\large 28.4} \\
+\bottomrule
+\end{tabular}
+\end{table}
+\end{center}
+
+\vspace{0.5cm}
+\begin{itemize}
+\item \highlight{Superior Training Speed:} Transformer (big) on En-Fr trained in only \textbf{3.5 days} on 8 P100 GPUs
+\item \highlight{Higher Parallelizability:} Attention computations are highly parallelizable
+\item \highlight{Lower Computational Cost:} Significantly fewer FLOPs for similar or better quality
+\end{itemize}
+\end{frame}
+
+% Slide 13: Generalization to Parsing
+\begin{frame}
+\frametitle{Results: Generalization to Parsing}
+\begin{columns}
+\begin{column}{0.6\textwidth}
+\begin{itemize}
+\item \highlight{Task:} English Constituency Parsing (predicting syntactic structure)
+\vspace{0.3cm}
+\item \highlight{Dataset:} WSJ (Penn Treebank), limited training data setting
+\vspace{0.3cm}
+\item \highlight{Model:} 4-layer Transformer, minimal task-specific tuning
+\vspace{0.3cm}
+\item \highlight{Performance (F1 Score):}
+    \begin{itemize}
+    \item WSJ only training: \textbf{91.3 F1}
+    \item Semi-supervised: \textbf{92.7 F1}
+    \end{itemize}
+\vspace{0.3cm}
+\item \highlight{Significance:} Strong generalization with minimal adaptation
+\end{itemize}
+\end{column}
+\begin{column}{0.4\textwidth}
+% Example parse tree
+% Increased scale and font size for better readability
+\begin{tikzpicture}[scale=0.9, % Increased scale from 0.7 to 0.9
+    node distance=0.7cm, % Adjusted node distance
+    every node/.style={font=\large} % Set default font size for all nodes
+]
+\node (s) at (2,3) {S};
+\node (np1) at (1,2.2) {NP};
+\node (vp) at (3,2.2) {VP};
+\node (dt) at (0.5,1.4) {DT};
+\node (nn) at (1.5,1.4) {NN};
+\node (vbz) at (2.5,1.4) {VBZ};
+\node (np2) at (3.5,1.4) {NP};
+\node (the) at (0.5,0.6) {The};
+\node (cat) at (1.5,0.6) {cat};
+\node (sits) at (2.5,0.6) {sits};
+\node (mat) at (3.5,0.6) {mat};
+
+\draw[thick] (s) -- (np1);
+\draw[thick] (s) -- (vp);
+\draw[thick] (np1) -- (dt);
+\draw[thick] (np1) -- (nn);
+\draw[thick] (vp) -- (vbz);
+\draw[thick] (vp) -- (np2);
+\draw[thick] (dt) -- (the);
+\draw[thick] (nn) -- (cat);
+\draw[thick] (vbz) -- (sits);
+\draw[thick] (np2) -- (mat);
+\end{tikzpicture}
+\end{column}
+\end{columns}
+\end{frame}
+
+% Slide 14: Key Findings & Impact
+\begin{frame}
+\frametitle{Key Findings \& Impact}
+\begin{itemize}
+\item \highlight{Key Findings:}
+    \begin{enumerate}
+    \item \textbf{Attention is sufficient:} Recurrence/convolution not necessary for SOTA sequence modeling
+    \item \textbf{Superior Quality:} New SOTA on major MT benchmarks
+    \item \textbf{Efficiency:} More parallelizable, faster training, fewer FLOPs
+    \item \textbf{Generalizability:} Strong performance on non-translation tasks
+    \end{enumerate}
+\vspace{0.5cm}
+\item \highlight{Transformative Impact:}
+    \begin{itemize}
+    \item \textbf{Paradigm Shift:} Revolutionized NLP and sequence modeling
+    \item \textbf{Foundation for Modern LLMs:} Paved the way for BERT, GPT, T5, etc.
+    \item \textbf{Enabled Scaling:} Facilitated training of much larger models on vast datasets
+    \end{itemize}
+\end{itemize}
+\end{frame}
+
+% Slide 15: Conclusion
+\begin{frame}
+\frametitle{Conclusion}
+\begin{itemize}
+\item The Transformer introduced a novel architecture based \textit{solely} on attention, successfully dispensing with recurrence and convolutions.
+\vspace{0.5cm}
+\item It demonstrated significant improvements in:
+    \begin{itemize}
+    \item \highlight{Translation Quality} (new SOTA BLEU scores)
+    \item \highlight{Training Efficiency} (faster, more parallelizable)
+    \item \highlight{Generalization} (strong parsing results)
+    \end{itemize}
+\vspace{0.5cm}
+\item This work marked a pivotal moment, establishing attention as a primary mechanism for sequence modeling and unlocking new possibilities in AI.
+\end{itemize}
+\end{frame}
+
+% Slide 16: Future Directions & Q&A
+\begin{frame}
+\frametitle{Future Directions \& Q\&A}
+\begin{itemize}
+\item \highlight{Future Work (Proposed by Authors):}
+    \begin{itemize}
+    \item Apply Transformer to other modalities (images, audio, video)
+    \item Explore local/restricted attention mechanisms for very long sequences
+    \item Investigate non-auto-regressive generation for further speed-ups
+    \end{itemize}
+\end{itemize}
+
+\vspace{1cm}
+\begin{center}
+\Large{\highlight{Thank You!}}
+\vspace{0.5cm}
+
+\Large{\highlight{Questions?}}
+\end{center}
+\end{frame}
+
+\end{document}