<p id="2eelq6757" class="D-Nq3 _5LB9L" data-pm-slice="1 1 []"><span style="font-weight: bold;"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color"><span style="background-color: null; text-decoration: inherit;" data-hook="background-color">Data Science Course in Pune</span></span>

<p id="zapxu6760" class="D-Nq3 _5LB9L">

<p id="twnz66761" class="D-Nq3 _5LB9L">

<p id="o1huf6762" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color">In a Transformer model, each word can consider the other words in a sentence, regardless of where they are located. The &ldquo;self-attention&rdquo; component is used to achieve this. Self-attention assigns each word a score based on its importance in relation to the other words. In the sentence, &ldquo;The cat sat upon the mat&rdquo;, the word &ldquo;cat&rdquo;, for example, might pay more attention to the words &ldquo;sat&rdquo; or &ldquo;mat&rdquo; rather than &ldquo;the&rdquo; to help the model better understand the meaning of the sentence.</span>

<p id="rnliq6764" class="D-Nq3 _5LB9L">

<p id="ionxg6765" class="D-Nq3 _5LB9L">

<p id="avba76766" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color">This process begins by converting the input tokens to three vectors: key and value. These vectors are generated by multiplying word embeddings and learned matrices. The dot product between the key and query of a word is used to compute the attention score. The scores are divided by the square roots of the dimensions of the key vectors for numerical stability, and then normalized into probabilities using a softmax. These weights for attention are then used to calculate a weighted total of the value vectors. This is the output of each word&rsquo;s attention layer.</span>

<p id="lrky26768" class="D-Nq3 _5LB9L">

<p id="wbbji6769" class="D-Nq3 _5LB9L">

<p id="dycne6770" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color"><span style="background-color: null; text-decoration: inherit;" data-hook="background-color">text.</span></span><span style="font-weight: bold;"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color"><span style="background-color: null; text-decoration: inherit;" data-hook="background-color">Data</span></span><span style="font-weight: bold;"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color"><span style="background-color: null; text-decoration: inherit;" data-hook="background-color"> Science Course in Pune</span></span></span>

<p id="bjom06775" class="D-Nq3 _5LB9L">

<p id="9ukio6776" class="D-Nq3 _5LB9L">

<p id="js1b46777" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color">The attention-based processing is repeated in multiple layers of both the decoder and encoder components. The encoder&rsquo;s attention layers learn the contextual representations for each word. Attention plays two roles in the decoder: self-attention layers let the decoder consider previous words to produce the next word and encoder-decoder layers help the coder focus on relevant parts of input sentences.</span>

<p id="i179f6779" class="D-Nq3 _5LB9L">

<p id="pdb7h6780" class="D-Nq3 _5LB9L">

<p id="wc9w06781" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color">Attention is a critical component of the model because it can handle longer-range dependencies more effectively than older models, such as RNNs or LSTMs. These earlier models struggled to retain context over long sequences. Attention eliminates the bottleneck that sequential processing creates, as every word interacts with every other.</span>

<p id="6pa7f6783" class="D-Nq3 _5LB9L">

<p id="awxwl6784" class="D-Nq3 _5LB9L">

<p id="wvugd6785" class="D-Nq3 _5LB9L"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color">Transformers also include , a positional encoding that addresses the lack of word order awareness inherent in the attention mechanism. The input embeddings are enhanced with positional encodings that provide information on the relative or absolute positions of words within a sequence. The model can maintain order and still process all tokens simultaneously.</span>

<p id="1rrgl6787" class="D-Nq3 _5LB9L">

<p id="waezj6788" class="D-Nq3 _5LB9L">

<p id="tdxl26789" class="D-Nq3 _5LB9L"><span style="font-weight: bold;"><span style="color: null; text-decoration: inherit;" data-hook="foreground-color"><span style="background-color: null; text-decoration: inherit;" data-hook="background-color">Data Science Course in Pune</span></span>