Difference between revisions of "My codes"

From Biolecture.org
imported>SeoGaLam
(Created page with "<p><span style="font-size:20px"><strong>Needleman-Wunsch algorithm</strong></span></p> <hr /> <p>The <strong>Needleman–Wunsch algorithm</strong> is an <a hr...")
(No difference)

Revision as of 02:53, 7 December 2018

Needleman-Wunsch algorithm

The Needleman–Wunsch algorithm is an algorithm used in bioinformatics to align protein or nucleotide sequences. It was one of the first applications of dynamic programming to compare biological sequences. The algorithm was developed by Saul B. Needleman and Christian D. Wunsch and published in 1970. The algorithm essentially divides a large problem (e.g. the full sequence) into a series of smaller problems and uses the solutions to the smaller problems to reconstruct a solution to the larger problem. It is also sometimes referred to as the optimal matching algorithm and the global alignment technique. The Needleman–Wunsch algorithm is still widely used for optimal global alignment, particularly when the quality of the global alignment is of the utmost importance.

Figure 1: Needleman-Wunsch pairwise sequence alignment

Results:

Sequences    Best alignments
---------    ----------------------
GCATGCU      GCATG-CU      GCA-TGCU      GCAT-GCU
GATTACA      G-ATTACA      G-ATTACA      G-ATTACA

Interpretation of the initialization step:

One can interpret the leftmost column in the above figure like this (putting a "handle" before each sequence, annotated as + here):

Alignment:  +GCATGCU
            +GATTACA
Score:      0  // Handle matches handle, doesn't win any score

Alignment:  +GCATGCU
           +GATTACA
Score:      0  // 1 gap,  score -1

Alignment:  +GCATGCU
          +GATTACA
Score:      0  // 2 gaps, score -2

Alignment:  +GCATGCU
         +GATTACA
Score:      0  // 3 gaps, score -3

Alignment:  +GCATGCU
        +GATTACA
Score:      0  // 4 gaps, score -4

...

The same thing can be done for the uppermost row.

 

Template:Technical
{{Infobox algorithm
|name= <!-- Defaults to article name -->
|class= Sequence alignment
|image=
|caption=
|data=
|time= <math>O(mn)</math>
|best-time=
|average-time=
|space= <math>O(mn)</math>
}}

The '''Needleman–Wunsch algorithm''' is an algorithm used in  bioinformatics to align protein or nucleotide sequences. It was one of the first applications of dynamic programming to compare biological sequences. The algorithm was developed by Saul B. Needleman and Christian D. Wunsch and published in 1970.<ref name=Needleman>Template:Cite journal</ref> The algorithm essentially divides a large problem (e.g. the full sequence) into a series of smaller problems and uses the solutions to the smaller problems to reconstruct a solution to the larger problem.<ref>Template:Cite web</ref> It is also sometimes referred to as the optimal matching algorithm and the global alignment technique. The Needleman–Wunsch algorithm is still widely used for optimal global alignment, particularly when the quality of the global alignment is of the utmost importance.

Figure 1: Needleman-Wunsch pairwise sequence alignment

<pre>
Results:

Sequences    Best alignments

   ----------------------
GCATGCU      GCATG-CU      GCA-TGCU      GCAT-GCU
GATTACA      G-ATTACA      G-ATTACA      G-ATTACA

Interpretation of the initialization step:

One can interpret the leftmost column in the above figure like this (putting a "handle" before each sequence, annotated as + here):

Alignment:  +GCATGCU
            +GATTACA
Score:      0  // Handle matches handle, doesn't win any score

Alignment:  +GCATGCU
           +GATTACA
Score:      0  // 1 gap,  score -1

Alignment:  +GCATGCU
          +GATTACA
Score:      0  // 2 gaps, score -2

Alignment:  +GCATGCU
         +GATTACA
Score:      0  // 3 gaps, score -3

Alignment:  +GCATGCU
        +GATTACA
Score:      0  // 4 gaps, score -4

...

The same thing can be done for the uppermost row.
</pre>



Template:-

==Introduction==
This algorithm can be used for any two strings. This guide will use two small DNA sequences as examples as shown in the diagram:
 GCATGCU
 GATTACA

===Constructing the grid===
First construct a grid such as one shown in Figure 1 above. Start the first string in the top of the third column and start the other string at the start of the third row. Fill out the rest of the column and row headers as in Figure 1. There should be no numbers in the grid yet.

{| class="wikitable"
|-
!|  ||   || G || C || A || T || G || C || U
|-
! scope="row" |
| &nbsp; ||  ||  ||  ||  ||  ||  ||
|-
! scope="row" | G
|  ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
|  ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | T
|  ||   ||   ||   ||   ||   ||   ||  
|-
! scope="row" | T
|  ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
|  ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | C
|  ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
|  ||   ||   ||   ||   ||   ||   || 
|}

===Choosing a scoring system===
Next, decide how to score each individual pair of letters. Using the example above, one possible alignment candidate might be:
Template:DNA sequence
The letters may match, mismatch, or be matched to a gap (a deletion or insertion (indel):

  • Match: The two letters at the current index are the same.
  • Mismatch: The two letters at the current index are different.  
  • Indel (INsertion or DELetion): The best alignment involves one letter aligning to a gap in the other string.

Each of these scenarios is assigned a score and the sum of the score of each pairing is the score of the whole alignment candidate. Different systems exist for assigning scores; some have been outlined in the Scoring systems section below. For now, the system used by Needleman and Wunsch<ref name="Needleman"/> will be used:

* Match: +1

  • Mismatch or Indel: −1

For the Example above, the score of the alignment would be 0:
Template:DNA sequence
 +-++--+- -> -1*4 + 1*4 = 0

===Filling in the table===
Start with a zero in the second row, second column. Move through the cells row by row, calculating the score for each cell. The score is calculated by comparing the scores of the cells neighboring to the left, top or top-left (diagonal) of the cell and adding the appropriate score for match, mismatch or indel. Calculate the candidate scores for each of the three possibilities:

  • The path from the top or left cell represents an indel pairing, so take the score of the left and the top cell, and add the score for indel to each of them.
  • The diagonal path represents a match/mismatch, so take the score of the top-left diagonal cell and add the score for match if the corresponding bases in the row and column are matching or the score for mismatch if they do not.

The resulting score for the cell is the highest of the three candidate scores.

Given there is no 'top' or 'top-left' cells for the second row only the existing cell to the left can be used to calculate the score of each cell. Hence -1 is added for each shift to the right as this represents an indelible from the previous score. This results in the first row being 0, -1, -2, -3, -4, -5, -6, -7. The same applies to the second column as only the existing score above each cell can be used. Thus the resulting table is:

{| class="wikitable"
|-
!|  ||   || G || C || A || T || G || C || U
|-
! scope="row" |
| 0 || -1 || -2 || -3 || -4 || -5 || -6 || -7
|-
! scope="row" | G
| -1 ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
| -2 ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | T
| -3 ||   ||   ||   ||   ||   ||   ||  
|-
! scope="row" | T
| -4 ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
| -5 ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | C
| -6 ||   ||   ||   ||   ||   ||   || 
|-
! scope="row" | A
| -7 ||   ||   ||   ||   ||   ||   || 
|}

The first case with existing scores in all 3 directions is the intersection of our first letters (in this case G and G). The surrounding cells are below:

    G

 0 -1
G
 -1 '''X'''

This cell has three possible candidate sums:

* The diagonal top-left neighbor has score 0. The pairing of G and G is a match, so add the score for match: 0+1 = 1

  • The top neighbor has score -1 and moving from there represents an indel, so add the score for indel: (-1) + (-1) = (-2)
  • The left neighbor also has score -1, represents an indel and also produces (-2).

The highest candidate is 1 and is entered into the cell:

{| class="wikitable"
|-
!|  ||   || G
|-
! scope="row" |
| 0 || -1
|-
! scope="row" | G
| -1 || '''1'''
|}

The cell which gave the highest candidate score must also be recorded. In the completed diagram in figure 1 above, this is represented as an arrow from the cell in row and column 3 to the cell in row and column 2.

In the next example, the diagonal step for both X and Y represents a mismatch:

{| class="wikitable"
|-
!|  || || G || C
|-
! scope="row" |
| 0 || -1 || -2
|-
! scope="row" | G
| -1 || 1 || '''X'''
|-
! scope="row" | A
| -2 || '''Y''' || 
|}

X:

  • Top: (-2)+(-1) = (-3)
  • Left: (+1)+(-1) = (0)
  • Top-Left: (-1)+(-1) = (-2)

Y:

  • Top: (1)+(-1) = (0)
  • Left: (-2)+(-1) = (-3)
  • Top-Left: (-1)+(-1) = (-2)

For both X and Y, the highest score is zero:

{| class="wikitable"
|-
!|  || || G || C
|-
! scope="row" |
| 0 || -1 || -2
|-
! scope="row" | G
| -1 || 1 || '''0'''
|-
! scope="row" | A
| -2 || '''0''' || 
|}

The highest candidate score may be reached by two or all neighboring cells:

{| class="wikitable"
|-
!|  || T || G
|-
! scope="row" | T
| 1 || 1
|-
! scope="row" | A
| 0 || '''X'''
|}

* Top: (1)+(-1) = (0)

  • Top-Left: (1)+(-1) = (0)
  • Left: (0)+(-1) = (-1)

In this case, all directions reaching the highest candidate score must be noted as possible origin cells in the finished diagram in figure 1, e.g. in the cell in row and column 7.

Filling in the table in this manner gives the scores or all possible alignment candidates, the score in the cell on the bottom right represents the alignment score for the best alignment.

===Tracing arrows back to origin===
Mark a path from the cell on the bottom right back to the cell on the top left by following the direction of the arrows. From this path, the sequence is constructed by these rules:

  • A diagonal arrow represents a match or mismatch, so the letters of the column and the letter of the row of the origin cell will align.
  • A horizontal or vertical arrow represents an indel. Horizontal arrows will align a gap ("-") to the letter of the row (the "side" sequence), vertical arrows will align a gap to the letter of the column (the "top" sequence).
  • If there are multiple arrows to choose from, they represent a branching of the alignments. If two or more branches all belong to paths from the bottom left to the top right cell, they are equally viable alignments. In this case, note the paths as separate alignment candidates.

Following these rules, the steps for one possible alignment candidate in figure 1 are:

 U → CU → GCU → -GCU → T-GCU → AT-GCU → CAT-GCU → '''GCAT-GCU'''
 A → CA → ACA → TACA → TTACA → ATTACA → -ATTACA → '''G-ATTACA'''
              ↓
     (branch) → TGCU → ...
              → TACA → ...

==Scoring systems==

===Basic scoring schemes===
The simplest scoring schemes simply give a value for each match, mismatch and indel. The step-by-step guide above uses match = 1, mismatch  = −1, indel = −1. Thus the lower the alignment score the larger the edit distance, for this scoring system one wants a high score. Another scoring system might be:

  • Match = 0
  • Indel = 1
  • Mismatch = 1

For this system the alignment score will represent the edit distance between the two strings.
Different scoring systems can be devised for different situations, for example if gaps are considered very bad for your alignment you may use a scoring system that penalises gaps heavily, such as:

  • Match = 0
  • Mismatch = 1
  • Indel = 10

Try it out.

===Similarity matrix===
More complicated scoring systems attribute values not only for the type of alteration, but also for the letters that are involved. For example, a match between A and A may be given 1, but a match between T and T may be given 4. Here (assuming the first scoring system) more importance is given to the Ts matching than the As, i.e. the Ts matching is  assumed to be more significant to the alignment. This weighting based on letters also applies to mismatches.

In order to represent all the possible combinations of letters and their resulting scores  a similarity matrix is used. The similarity matrix for the most basic system is represented as:

{| style="font-family: monospace; font-size: 150%;" cellpadding="2" class="wikitable"
! scope="col" |
! scope="col" | A
! scope="col" | G
! scope="col" | C
! scope="col" | T
|- style="text-align: right;"
! scope="row" | A
| 1 ||  -1|| -1 || -1
|- style="text-align: right;"
! scope="row" | G
| -1 || 1 || -1 || -1
|- style="text-align: right;"
! scope="row" | C
| -1 || -1 || 1 || -1
|- style="text-align: right;"
! scope="row" | T
| -1 || -1 || -1 ||  1
|}
Each score represents a switch from one of the letters the cell matches to the other. Hence this represents all possible matches and deletions (for an alphabet of ACGT). Note all the matches go along the diagonal, also not all the table needs to be filled, only this triangle because the scores are reciprocal.= (Score for A → C = Score for C → A). If implementing the T-T = 4 rule from above the following similarity matrix is produced:

{| style="font-family: monospace; font-size: 150%;" cellpadding="2" class="wikitable"
! scope="col" |
! scope="col" | A
! scope="col" | G
! scope="col" | C
! scope="col" | T
|- style="text-align: right;"
! scope="row" | A
| 1 || -1 || -1 || -1
|- style="text-align: right;"
! scope="row" | G
| -1 || 1 || -1 || -1
|- style="text-align: right;"
! scope="row" | C
| -1 || -1 || 1 || -1
|- style="text-align: right;"
! scope="row" | T
| -1 || -1 || -1 ||  4
|}

Different scoring matrices have been statistically constructed which give weight to different actions appropriate to a particular scenario. Having weighted scoring matrices is particularly important in protein sequence alignment due to the varying frequency of the different amino acids. There are two broad families of scoring matrices, each with further alterations for specific scenarios:

===Gap penalty===
When aligning sequences there are often gaps (i.e. indels), sometimes large ones. Biologically, a large gap is more likely to occur as one large deletion as opposed to multiple single deletions. Hence two small indels should have a worse score than one large one. The simple and common way to do this is via a large gap-start score for a new indel and a smaller gap-extension score for every letter which extends the indel. For example, new-indel may cost -5 and extend-indel may cost -1. In this way an alignment such as:
 GAAAAAAT
 G--A-A-T
which has multiple equal alignments, some with multiple small alignments will now align as:
 GAAAAAAT
 GAA----T
or any alignment with a 4 long gap in preference over multiple small gaps.

==Advanced presentation of algorithm==
Scores for aligned characters are specified by a similarity matrix. Here, Template:Math is the similarity of characters ''a'' and ''b''. It uses a linear gap penalty, here called Template:Mvar.

For example, if the similarity matrix was


 A
 G
 C
 T
A
 10 -1 -3 -4
G
 -1 7 -5 -3
C
 -3 -5 9 0
T
 -4 -3 0 8

then the alignment:
 AGACTAGTTAC
 CGA---GACGT
with a gap penalty of -5, would have the following score:

Template:Math
= -3 + 7 + 10 - (3 × 5) + 7 + (-4) + 0 + (-1) + 0 = 1

To find the alignment with the highest score, a two-dimensional array (or matrix) ''F'' is allocated. The entry in row ''i'' and column ''j'' is denoted here by
<math>F_{ij}</math>. There is one row for each character in sequence ''A'', and one column for each character in sequence ''B''. Thus, if aligning sequences of sizes ''n'' and ''m'', the amount of memory used is in <math>O(nm)</math>. Hirschberg's algorithm only holds a subset of the array in memory and uses <math>\Theta(\min \{n,m\})</math> space, but is otherwise similar to Needleman-Wunsch (and still requires <math>O(nm)</math> time).

As the algorithm progresses, the <math>F_{ij}</math> will be assigned to be the optimal score for the alignment of the first <math>i=0,\dotsc,n</math> characters in ''A'' and the first <math>j=0,\dotsc,m</math> characters in ''B''. The principle of optimality is then applied as follows:

  • Basis:
<math>F_{0j} = d*j</math>
<math>F_{i0} = d*i</math>
  • Recursion, based on the principle of optimality:
<math>F_{ij} = \max(F_{i-1,j-1} + S(A_{i}, B_{j}), \; F_{i,j-1} + d, \; F_{i-1,j} + d)</math>

The pseudo-code for the algorithm to compute the F matrix therefore looks like this:
 d ← MismatchScore
 '''for''' i=0 '''to''' '''length'''(A)
   F(i,0) ← d*i
 '''for''' j=0 '''to''' '''length'''(B)
   F(0,j) ← d*j
 '''for''' i=1 '''to''' '''length'''(A)
   '''for''' j=1 '''to''' '''length'''(B)
   {
     Match ← F(i-1,j-1) + S(A<sub>i</sub>, B<sub>j</sub>)
     Delete ← F(i-1, j) + d
     Insert ← F(i, j-1) + d
     F(i,j) ← '''max'''(Match, Insert, Delete)
   }
Once the ''F'' matrix is computed, the entry <math>F_{nm}</math> gives the maximum score among all possible alignments. To compute an alignment that actually gives this score, you start from the bottom right cell, and compare the value with the three possible sources (Match, Insert, and Delete above) to see which it came from. If Match, then <math>A_i</math> and <math>B_j</math> are aligned, if Delete, then <math>A_i</math> is aligned with a gap, and if Insert, then <math>B_j</math> is aligned with a gap. (In general, more than one choice may have the same value, leading to alternative optimal alignments.)
 AlignmentA ← ""
 AlignmentB ← ""
 i ← '''length'''(A)
 j ← '''length'''(B)
 '''while''' (i > 0 '''or''' j > 0)
 {
   '''if''' (i > 0 '''and''' j > 0 '''and''' F(i,j) == F(i-1,j-1) + S(A<sub>i</sub>, B<sub>j</sub>))
   {
     AlignmentA ← A<sub>i</sub> + AlignmentA
     AlignmentB ← B<sub>j</sub> + AlignmentB
     i ← i - 1
     j ← j - 1
   }
   '''else''' '''if''' (i > 0 '''and''' F(i,j) == F(i-1,j) + d)
   {
     AlignmentA ← A<sub>i</sub> + AlignmentA
     AlignmentB ← "-" + AlignmentB
     i ← i - 1
   }
   '''else'''
   {
     AlignmentA ← "-" + AlignmentA
     AlignmentB ← B<sub>j</sub> + AlignmentB
     j ← j - 1
   }
 }

== Complexity ==
Computing the score <math>F_{ij}</math> for each cell in the table is an <math>O(1)</math> operation. Thus the time complexity of the algorithm for two sequences of length <math>n</math> and <math>m</math> is <math>O(mn)</math>.<ref name=":0">Template:Cite book</ref> It has been shown that it is possible to improve the running time to <math>O(mn/ \log n)</math> using the Method of Four Russians.<ref name=":0" /><ref>Template:Cite journal</ref> Since the algorithm fill an <math>n \times m</math> table the space complexity is <math>O(mn)</math>.<ref name=":0" />

==Historical notes and algorithm development==
The original purpose of the algorithm described by Needleman and Wunsch was to find similarities in the amino acid sequences of two proteins.<ref name=Needleman />

Needleman and Wunsch describe their algorithm explicitly for the case when the alignment is penalized solely by the matches and mismatches, and gaps have no penalty (''d''=0). The original publication from 1970 suggests the recursion
<math>F_{ij} = \max_{h<i,k<j} \{ F_{h,j-1}+S(A_{i},B_{j}), F_{i-1,k}+S(A_i,B_j) \}</math>.

The corresponding dynamic programming algorithm takes cubic time. The paper also points out that the recursion can accommodate arbitrary gap penalization formulas:

<blockquote>
A penalty factor, a number subtracted for every gap made, may be assessed as a barrier to allowing the gap. The penalty factor could be a function of the size and/or direction of the gap. [page 444]
</blockquote>

A better dynamic programming algorithm with quadratic running time for the same problem (no gap penalty) was first introduced<ref name=Sankoff>Template:Cite journal</ref> by David Sankoff in 1972.
Similar quadratic-time algorithms were discovered independently
by T. K. Vintsyuk<ref name=Vintsyuk>Template:Cite journal</ref> in 1968 for speech processing
("time warping"), and by Robert A. Wagner and Michael J. Fischer<ref name=WagnerFischer>Template:Cite journal</ref> in 1974 for string matching.

Needleman and Wunsch formulated their problem in terms of maximizing similarity. Another possibility is to minimize the edit distance between sequences, introduced by Vladimir Levenshtein. Peter H. Sellers showed<ref name=Sellers>Template:Cite journal</ref> in 1974 that the two problems are equivalent.

The Needleman–Wunsch algorithm is still widely used for optimal global alignment, particularly when the quality of the global alignment is of the utmost importance. However, the algorithm is expensive with respect to time and space, proportional to the product of the length of two sequences and hence is not suitable for long sequences.

Recent development has focused on improving the time and space cost of the algorithm while maintaining quality. For example, in 2013, a Fast Optimal Global Sequence Alignment Algorithm (FOGSAA),<ref>Template:Cite journal</ref> suggested alignment of nucleotide/protein sequences faster than other optimal global alignment methods, including the Needleman–Wunsch algorithm. The paper claims that when compared to the Needleman–Wunsch algorithm, FOGSAA achieves a time gain of 70–90% for highly similar nucleotide sequences (with > 80% similarity), and 54–70% for sequences having 30–80% similarity.

==Global alignment tools using the Needleman–Wunsch algorithm==

==Applications outside bioinformatics==

===Computer stereo vision===
Stereo matching is an essential step in the process of 3D reconstruction from a pair of stereo images. When images have been rectified, an analogy can be drawn between aligning nucleotide and protein sequences and matching pixels belonging to scan lines, since both tasks aim at establishing optimal correspondence between two strings of characters. The ‘right’ image of a stereo pair can be seen as a mutated version of the ‘left’ image: noise and individual camera sensitivity alter pixel values (i.e. character substitutions); and different view angle reveals previously occluded data and introduces new occlusions (i.e. insertion and deletion of characters). As consequence, minor modifications of the Needleman–Wunsch algorithm make it suitable for stereo matching.<ref>Dieny R., Thevenon J., Martinez-del-Rincon J., Nebel J.-C. (2011) "Bioinformatics inspired algorithm for stereo correspondence". International Conference on Computer Vision Theory and Applications, March 5–7, Vilamoura - Algarve, Portugal.</ref> Although performances in terms of accuracy are not state-of-the-art, the relative simplicity of the algorithm allows its implementation on embedded systems.<ref>Madeo S., Pelliccia R., Salvadori C., Martinez-del-Rincon J., Nebel J.-C. (2014) "An optimized stereo vision implementation for embedded systems: application to RGB and Infra-Red images". Journal of Real-Time Image Processing.</ref>

Although in many applications image rectification can be performed, e.g. by camera resectioning or calibration, it is sometimes impossible or impractical since the computational cost of accurate rectification models prohibit their usage in real-time applications. Moreover, none of these models is suitable when a camera lens displays unexpected distortions, such as those generated by raindrops, weatherproof covers or dust. By extending the Needleman–Wunsch algorithm, a line in the ‘left’ image can be associated to a curve in the ‘right’ image by finding the alignment with the highest score in a three-dimensional array (or matrix). Experiments demonstrated that such extension allows dense pixel matching between unrectified or distorted images.<ref>Martinez-del-Rincon J., Thevenon J., Dieny R., Nebel J.-C. (2012) "Dense Pixel Matching Between Unrectified and Distorted Images Using Dynamic Programming". International Conference on Computer Vision Theory and Applications, 24–26 February, Rome, Italy.</ref>

==See also==

==References==
Template:Reflist

==External links==
Template:External links

Template:Strings






Retrieved from "https://Biolecture.org/index.php?title=My_codes&oldid=12466"