Repository navigation
Expand file tree
/
Copy pathindex.xml
More file actions
1915 lines (1424 loc) · 85.7 KB
/
Copy pathindex.xml
File metadata and controls
1915 lines (1424 loc) · 85.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title><span class="top">Lucas</span> <span class="bottom">van Dijk</span> on <span class="top">Lucas</span> <span class="bottom">van Dijk</span></title>
<link>https://lucasvandijk.nl/</link>
<description>Recent content in <span class="top">Lucas</span> <span class="bottom">van Dijk</span> on <span class="top">Lucas</span> <span class="bottom">van Dijk</span></description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<copyright>&copy; 2018 Lucas van Dijk</copyright>
<lastBuildDate>Wed, 06 Sep 2017 00:00:00 +0000</lastBuildDate>
<atom:link href="/" rel="self" type="application/rss+xml" />
<item>
<title></title>
<link>https://lucasvandijk.nl/resume/about/</link>
<pubDate>Wed, 06 Sep 2017 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/resume/about/</guid>
<description>
<h1 id="about">About</h1>
<p>I am a PhD-candidate in bioinformatics in the <a href="https://www.tudelft.nl/ewi/over-de-faculteit/afdelingen/intelligent-systems/pattern-recognition-bioinformatics/the-delft-bioinformatics-lab/" target="_blank">Delft Bioinformatics
Lab</a> at TU Delft and my project is in close collaboration
with the <a href="https://www.broadinstitute.org/infectious-disease-and-microbiome/bacterial-genomics-group-people" target="_blank">Bacterial Genomics Group</a> at the Broad Institute
of MIT and Harvard, where I do most of my work. My background is in electrical
engineering and computer science, and I have gained a huge interest in biology
the past few years. Now I am using my knowledge on algorithms, data structures
and machine learning to help answer biological questions.</p>
<p>My research focusses on developing new tools and algorithms to analyse
bacterial populations. While many bacteria are harmless, some specific strains
can become pathogenic, even though they&rsquo;re considered the same species. The
goal is to improve our ability to track and dissect the genomes of specific
bacterial strains in (DNA) sequencing datasets. This will aid in our
understanding how antibiotic resistance arises in bacteria, or how the genes
responsible for antibiotic resistance spread within a patient or hospital.</p>
</description>
</item>
<item>
<title>Experience</title>
<link>https://lucasvandijk.nl/resume/professional/</link>
<pubDate>Tue, 27 Mar 2018 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/resume/professional/</guid>
<description>
<h2 id="professional-appointments">Professional appointments</h2>
<ul>
<li><strong>Computational Associate I</strong>, <em>Broad Institute of MIT and Harvard</em>,
Nov. 2017 - <em>present</em><br />
Develop algorithms and tools for strain-level metagenomics.</li>
<li><strong>PhD-candidate</strong>, <em>TU Delft</em>,
Nov. 2017 - <em>present</em><br />
See above.</li>
<li><strong>Bioinformatics intern</strong>, <em>DSM</em>,
Jun. 2016 - Oct. 2016<br />
Responsible for doing an initial analysis and literature review on a protein
production problem at large scale with the bacterium Bacillus subtilis by
modelling its proteome allocation and analysing protein measurements under
different growth conditions.</li>
<li><strong>Software engineer</strong>, <em>Studio Bereikbaar</em>,
May 2013 - May 2016<br />
Built several tools and applications to aid in data analysis of traffic engineering
problems. Using these tools and applications,my coworkers could collaborate on evaluating
multiple road construction scenarios quickly, which helped winning
several tenders for the Dutch government.</li>
</ul>
<h2 id="other-internships-side-jobs-and-projects">Other internships, side-jobs and projects</h2>
<ul>
<li><strong>Google summer of code student</strong>, <em>Python software foundation</em>,
May 2015 - Aug. 2015<br />
Implemented a graph visualisation system using OpenGL for the scientific
visualisation library <a href="http://vispy.org/" target="_blank">Vispy</a>.</li>
<li><strong>Embedded software developer</strong>, <em>TU Delft</em>,
Feb. 2012 - Feb. 2013<br />
Built a huge (3x3x1m, 24x24x8 LEDs) RGB LED display with a group of other students at Delft University of Technology. I was responsible for the embedded software on the Raspberry Pi.</li>
<li>Several other software engineering internships and jobs, 2009-2016</li>
</ul>
</description>
</item>
<item>
<title>Teaching</title>
<link>https://lucasvandijk.nl/resume/teaching/</link>
<pubDate>Tue, 27 Mar 2018 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/resume/teaching/</guid>
<description>
<h2 id="teaching-assistant">Teaching assistant</h2>
<p>I have been teaching assistant for the following undergraduate courses at TU
Delft:</p>
<ul>
<li>TI2736-C Data Mining (2016-2017)</li>
<li>NB2161 Bioinformatics (2016-2017)</li>
<li>LB2291 Bioinformatics (2016-2017)</li>
</ul>
</description>
</item>
<item>
<title>Software</title>
<link>https://lucasvandijk.nl/resume/projects/</link>
<pubDate>Wed, 20 Apr 2016 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/resume/projects/</guid>
<description></description>
</item>
<item>
<title>Using locality sensitive hashing to compactly represent k-mers</title>
<link>https://lucasvandijk.nl/2018/03/using-locality-sensitive-hashing-to-compactly-represent-k-mers/</link>
<pubDate>Mon, 26 Mar 2018 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/2018/03/using-locality-sensitive-hashing-to-compactly-represent-k-mers/</guid>
<description><p>Comparing (DNA) sequences is one of the core tasks in bioinformatics and the
classic approach is to <em>align</em> these sequences. This is, however, a relative
slow process and not always computationally feasible, especially if you want to
compare more than two DNA sequences. An alternative approach is compare
sequences based on their <em>k-mer profiles</em>.</p>
<p>A <em>k-mer</em> of a string $S$ is defined as any substring of $S$ of length $k$. For
example, the DNA sequence <code>AGCGTATCGATTCA</code> has the following k-mers if $k=6$:</p>
<pre><code>AGCGTATCGATTCA
--------------
AGCGTA
GCGTAT
CGTATC
...
GATTCA
</code></pre>
<p>As you can see, obtaining all <em>k-mers</em> is easy: slide a window of size $k$
along your sequence, yielding a k-mer at each position. A sequence of length
$L$ has $L - k + 1$ k-mers. A common task is to count how often each k-mer
occurs and compare genomes based on these counts. The main idea is
that similar genomes have similar <em>k-mer</em> counts<sup class="footnote-ref" id="fnref:1"><a href="#fn:1">1</a></sup>.</p>
<p>When dealing with the scale of genomes, storing counts for all these different
k-mers can take up quite a lot of memory. First, the number of distinct
<em>k-mers</em> grows exponentially with the length of $k$. In the case of DNA sequences,
our alphabet size is 4: A, C, T, G. Then there are $4^k$
possibilities of length $k$. The value of $k$
depends on your application and organism, but values ranging from 5 to 32 are
common.</p>
<p>Next, think how we would store the <em>k-mer</em> itself.
We could store each letter as ASCII character, requiring 8-bits per character.
This is a bit wasteful, however, because in DNA we only have 4 different
characters. An optimisation would be to use 2 bits per character: A=00, C=01,
T=10, G=11. This would allow us to store a <em>k-mer</em> of length 32 in a 64-bit
integer. Still, this may not be good enough. I&rsquo;ve seen cases for
$k=23$ where it went up to more than 100 GB, and that&rsquo;s quite a lot of memory
even if you have access to a decent compute cluster.</p>
<p>This post will explain a technique described in the paper by Cleary <em>et al.</em><sup class="footnote-ref" id="fnref:2"><a href="#fn:2">2</a></sup>
to reduce the memory consumption for storing <em>k-mers</em>. The main insight is that
we often don&rsquo;t need the <em>exact</em> count of each <em>k-mer</em>, and can take some
shortcuts by missing some <em>k-mers</em>. Because of the exponential number of
different <em>k-mers</em> and because genomes are often large, missing a few <em>k-mers</em>
will not have a huge impact. Furthermore, when dealing with whole genome
sequencing datasets, you also have to deal with sequencing error, and expect
some <em>k-mers</em> to be false. In a lot of cases using approximate <em>k-mer</em> counts is
appropriate.</p>
<p>We start by transforming each <em>k-mer</em> to a complex vector, using the following
encoding:</p>
<p>$$ A=1, T=-1, C=i, G=i $$</p>
<p>A <em>k-mer</em> is now a vector of length $k$ where each element represents a DNA base
as complex number. A <em>k-mer</em> is now a point in a $k$-dimensional space. Notice
that similar <em>k-mers</em> will be &ldquo;neighbours&rdquo; in this $k$-dimensional space. This is
not an efficient encoding however, but we can apply a few smart operations to
convert this to a smaller integer.</p>
<p>The idea is to divide our $k$-dimensional space into a lot of bins. This can be
done as follows: generate a random complex vector of length $k$, which is the normal
vector of plane through our $k$-dimensional space. This plane divides our space
in half: certain k-mers lie on the &ldquo;left&rdquo; side, while other k-mers lie on the
&ldquo;right&rdquo; side of this plane. See for an example the figure below.
If we note a 0 when the <em>k-mer</em> lies on the left,
and a 1 if on the right, we obtain a bit value.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/kmer-space-split.png" alt="
A few (poorly drawn) examples of splitting a $k$-dimensional space in half
by a hyperplane.
" />
<figcaption>
A few (poorly drawn) examples of splitting a $k$-dimensional space in half
by a hyperplane.
</figcaption>
</figure>
</p>
<p>Repeat this $n$ times, and you obtain $n$ bit values. For example, we could
store the result from drawing 32 random hyperplanes in a 32-bit integer. Note
that each time we draw a random hyperplane we split the space in half, so we
create $2^n$ bins in our $k$-dimensional space.</p>
<p>It is possible, however, that two distinct <em>k-mers</em> end up in the same bin,
and thus have the same 32-bit integer value (if $n=32$). We can actually
calculate the probability of such event. Recall that our <em>k-mers</em> are just
ordinary complex vectors, and that similar <em>k-mers</em> are neighbours in the
$k$-dimensional space. We can compute the cosine of the angle between the
two vectors as follows:</p>
<p>$$ \phi = \text{cos}(\theta)
= \frac{\textbf{u}\cdot\textbf{v}}{||\textbf{u}|| ||\textbf{v}||} $$</p>
<p>Here <strong>u</strong> and <strong>v</strong> are two <em>k-mers</em> in our complex $k$-dimensional space.
The probability of these two <em>k-mers</em> being in the same bin, is equal to the
probability that <em>no</em> random hyperplane cuts through the angle of <strong>u</strong> and
<strong>v</strong>, this is visualised in the figure below.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/kmer-neighbours.png" alt="
The probability that a hyperplane cuts through the angle of two
similar k-mers is lower than with two very dissimilar k-mers.
" />
<figcaption>
The probability that a hyperplane cuts through the angle of two
similar k-mers is lower than with two very dissimilar k-mers.
</figcaption>
</figure>
</p>
<p>We end up with the following equation:</p>
<p>$$ P(h(\textbf{u}) = h(\textbf{v})) = 1 - \frac{\text{acos}(\phi)}{\pi} $$</p>
<p>Here $h(\textbf{u})$ represent the <em>hash</em> value of a <em>k-mer</em> <strong>u</strong>, or the $n$-bit
integer by generating $n$ random hyperplanes.</p>
<p>This is the reason why this could be used for <em>approximate</em> counting of
<em>k-mers</em>, because some <em>k-mers</em> may be mistaken for another. By tuning $k$ and
$n$ you can try to minimise the impact on your application. The impact is even
further minimised because similar <em>k-mers</em> have a higher probability of ending
up in the same bin than dissimilar <em>k-mers</em>, while reducing the number of bits
you need to store the <em>k-mer</em>.</p>
<div class="footnotes">
<hr />
<ol>
<li id="fn:1">Zielezinski, A., Vinga, S., Almeida, J. &amp; Karlowski, W. M. <em>Alignment-free sequence comparison: Benefits, applications, and tools.</em> Genome Biol. 18, 1–17 (2017).
<a class="footnote-return" href="#fnref:1"><sup>^</sup></a></li>
<li id="fn:2">Cleary, B. et al. <em>Detection of low-abundance bacterial strains in metagenomic datasets by eigengenome partitioning.</em> Nat. Biotechnol. 33, 1053–1060 (2015).
<a class="footnote-return" href="#fnref:2"><sup>^</sup></a></li>
</ol>
</div></description>
</item>
<item>
<title>Visualising the height of the Netherlands</title>
<link>https://lucasvandijk.nl/2018/02/visualising-the-height-of-the-netherlands/</link>
<pubDate>Thu, 01 Feb 2018 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/2018/02/visualising-the-height-of-the-netherlands/</guid>
<description><p>On this date 65 years ago, February 1st 1953, the Netherlands experienced its
greatest flood till date, the <a href="https://en.wikipedia.org/wiki/North_Sea_flood_of_1953" target="_blank">North Sea Flood of 1953</a>. This
is still one of the biggest disasters the Netherlands has ever experienced,
with thousands of casualties and lots of people who lost their homes.</p>
<p>The Netherlands earns its name from the fact that large parts of the country
lie below sea level. To make sure our country doesn&rsquo;t flood, we have built lots
of barriers, dams and dykes to keep the water out, and we want to prevent
anything like the flood of 1953 from happening ever again.</p>
<p>In the beginning of this year, several of these barriers and dams were put to
the test when a heavy storm reached the Netherlands which resulted in very high
water levels. Our five biggest dams and barriers needed to be closed at the
same time, a first since their construction. We can ask ourselves the question
whether this will happen more often now that sea levels are rising due to
global warming <sup class="footnote-ref" id="fnref:sealevel"><a href="#fn:sealevel">1</a></sup>. A higher base line sea level increases the chance
for even higher water levels when it storms. To get an idea what areas would be
affected the most by a possible flood, we have created a visualisation project
that shows the height of the Netherlands in comparison to the sea level.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/maeslantkering.jpg" alt="Maeslantkering" />
<figcaption>
The Maeslantkering in closed state.
Photo: <a href="https://beeldbank.rws.nl" target="_blank">Rijkswaterstaat</a>
</figcaption>
</figure>
</p>
<h2 id="things-to-check-out">Things to check out</h2>
<ul>
<li>The Y-axis shows deviation from &ldquo;<a href="https://en.wikipedia.org/wiki/Amsterdam_Ordnance_Datum" target="_blank">Amsterdam Ordnance Datum</a>&ldquo;</li>
<li>You can zoom on both the map and the elevation profile (by selecting a region
in the bottom profile).</li>
<li>If you hover your mouse over the elevation profile, a red dot will show you
the location on the map</li>
<li>The highest point in our country is the <a href="https://nl.wikipedia.org/wiki/Vaalserberg" target="_blank">Vaalserberg</a>, with a
whopping 322 m! It&rsquo;s in the most southern part of the Netherlands. Some
people, however, may claim it&rsquo;s not really part of the Netherlands because of
their ugly accents ;)</li>
<li>If you zoom in on our shorelines, you can see our dykes</li>
</ul>
<h2 id="title">Elevation profiles of the Netherlands</h2>
<div id="chart"></div>
<div id="map"></div>
<h2 id="technical-details">Technical details</h2>
<h3 id="quick-links">Quick links</h3>
<ul>
<li>Data source: <a href="http://ahn.nl" target="_blank">http://ahn.nl</a>
<ul>
<li>We use AHN version 2 and the dataset with a resolution of 5 meter</li>
</ul></li>
<li>Written in Python and CoffeeScript, and available on Github:
<a href="https://github.com/lrvdijk/nl-height" target="_blank">https://github.com/lrvdijk/nl-height</a></li>
</ul>
<h3 id="algorithm-overview">Algorithm overview</h3>
<ol>
<li>The publicly available height data is split across multiple &ldquo;raster data&rdquo;
files. The AHN website provides a &ldquo;ahn_units.shp&rdquo; shapefile, that shows
which part of the Netherlands is covered by which <em>raster data</em> file, and
thus shows which file to download.</li>
<li>We import this ahn_units shapefile in a PostgreSQL database with the PostGIS
extension enabled. A script downloads and extracts all available raster data
files.</li>
<li>When all data is downloaded, we import the raster data in our PostgreSQL
database.</li>
<li>To generate an elevation profile, we draw a line horizontally across the
Netherlands and query which cells in our raster data hit our line. Each cell
contains the height at its midpoint. Our query is based on this blogpost:
<a href="http://blog.mathieu-leplatre.info/drape-lines-on-a-dem-with-postgis.html" target="_blank">http://blog.mathieu-leplatre.info/drape-lines-on-a-dem-with-postgis.html</a></li>
<li>When there&rsquo;s water, no data exists in our database. We apply some
post-processing such that our visualisation does not get weird artefacts.</li>
<li>We store the data in a JSON file, and use the d3.js library for the
visualisation.</li>
</ol>
<div class="footnotes">
<hr />
<ol>
<li id="fn:sealevel"><a href="https://climate.nasa.gov/vital-signs/sea-level/" target="_blank">https://climate.nasa.gov/vital-signs/sea-level/</a>
<a class="footnote-return" href="#fnref:sealevel"><sup>^</sup></a></li>
</ol>
</div></description>
</item>
<item>
<title>PHASM: Haplotype-aware de novo genome assembly</title>
<link>https://lucasvandijk.nl/project/phasm/</link>
<pubDate>Tue, 29 Aug 2017 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/project/phasm/</guid>
<description>
<p>PHASM is a long read <em>de novo</em> genome assembler that phases variants among
chromosome homologues during the assembly process, and aims to output separate
contigs for each haplotype. The main idea in PHASM is to build bubble chains:
consecutive &ldquo;superbubbles&rdquo; chained together. While most traditional genome
assemblers pop these superbubbles by only keeping the best supported path,
PHASM finds <em>k</em> paths through this chain of superbubbles that best represent
each haplotype.</p>
<p>This program has been created as part of my master thesis project. For now, it
has only been tested with error free data.</p>
<ul>
<li><strong>Status:</strong> Finished.</li>
<li><strong>License</strong>: MIT</li>
</ul>
<h2 id="overview">Overview</h2>
<h3 id="pipeline">Pipeline</h3>
<p>The PHASM pipeline consists of four main stages:</p>
<ol>
<li>Overlapper</li>
<li>Assembly graph construction</li>
<li>Bubblechain identification</li>
<li>Phasing</li>
</ol>
<p><img src="https://lucasvandijk.nl/img/projectimages/phasm-overview.png" alt="PHASM pipeline overview" /></p>
<h3 id="key-points">Key Points</h3>
<ul>
<li>Main idea:
<ul>
<li>Different alleles on homologous chromosomes result in branches in the
assembly graph;</li>
<li>A bubble or <em>superbubble</em> is a common motif in a genome assembly graph;</li>
<li><strong>Hypothesis</strong>: each superbubble has a valid path from its source to its
sink that spells a DNA sequence that matches with one of the haplotypes.
Furthermore, we expect that for each haplotype, there exists at least one
path through this superbubble that spells a matching DNA sequence.</li>
<li>A lot of current genome assemblers pop bubbles in the assembly graph,
thereby removing variation among homologous chromosomes</li>
<li>Build an assembler that keeps superbubbles and searches for the best set
of paths such that each path represents a haplotype.</li>
</ul></li>
<li>Main findings:
<ul>
<li>Superbubbles are an incomplete representation of variation among
homologous chromosomes</li>
<li>Paths through a superbubble are rarely consistent with one of the
haplotypes</li>
<li>Superbubbles are probably an artefact of approximate overlapping between
reads.</li>
</ul></li>
</ul>
<p>Read more in my <a href="https://lucasvandijk.nl/docs/master-thesis.pdf">master thesis</a>.</p>
<h2 id="availability">Availability</h2>
<p>PHASM is available on <a href="https://github.com/AbeelLab/phasm" target="_blank">Github</a>.</p>
</description>
</item>
<item>
<title>Drawing arbitrary shapes with OpenGL points</title>
<link>https://lucasvandijk.nl/2015/06/drawing-arbitrary-shapes-with-opengl-points/</link>
<pubDate>Sat, 13 Jun 2015 00:00:00 +0000</pubDate>
<guid>https://lucasvandijk.nl/2015/06/drawing-arbitrary-shapes-with-opengl-points/</guid>
<description><p>Part of my Google Summer of Code project involves porting several arrow
heads from <a href="https://github.com/glumpy/glumpy" target="_blank">Glumpy</a> to
<a href="https://vispy.org" target="_blank">Vispy</a>. I also want to make a slight change to them:
the arrow heads in Glumpy include an arrow body, I want to remove that
to make sure you can put an arrow head on every type of line you want.</p>
<p>Making a change like that requires that you understand how those shapes
are drawn. And for someone without a background in computer graphics this took some thorough
investigation of the code and the techniques used. This article is aimed
at people like me: good enough programming skills and linear algebra
knowledge, but almost no former experience with OpenGL or computer
graphics in general.</p>
<h2 id="implicit-surfaces">Implicit surfaces</h2>
<p>The basic principle behind drawing these 2D shapes is called implicit
surfaces, and it relies on a arbitrary shape function which for a given
point in your image returns the distance to your shape surface or
boundary from that given point. This is visualized in fig. 1.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/implicit-surfaces.png" alt="
Distances to the boundary of a circle
" />
<figcaption>
Distances to the boundary of a circle
</figcaption>
</figure>
</p>
<p>The distances any shape function should return are highlighted in red.
To actually be able to draw a shape, we need to distinguish whether a
point lies inside the shape or not. We make the arbitrary decision that
a negative distance lies inside a shape, and a positive distance means
that the point lies outside the shape.</p>
<h2 id="distance-functions-for-a-few-basic-shapes">Distance functions for a few basic shapes</h2>
<p>The distance functions defined below have one requirement: the center
point of the shape has the coordinate (0, 0).</p>
<h3 id="circle">Circle</h3>
<p>For a circle these distances are easily calculated:</p>
<p>$$ d(\textbf{x}) = ||\textbf{x}|| − r $$</p>
<p>Where:</p>
<ul>
<li>$\textbf{x}$: Vector to the point in consideration.</li>
<li>$r$: The radius of the circle.</li>
</ul>
<p>If the point lies within the circle, the length of the vector towards
that point is smaller than the radius, making the distance automatically
negative.</p>
<h3 id="square">Square</h3>
<p>Consider fig. 2.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/distance-square.png" alt="
Visualizing the distances to the border of a square
" />
<figcaption>
Visualizing the distances to the border of a square
</figcaption>
</figure>
</p>
<p>A square is a nice symmetric figure, so we can take the absolute values
of the coordinates, and then we only need to consider the smaller
highlighted square (light blue).</p>
<p>The distance to the boundary of the square is then:</p>
<p>$$ d(\textbf{x}) = \text{max}(|x_1|, |x_2|) - \frac{s}{2}$$</p>
<p>Where:</p>
<ul>
<li>$\textbf{x}$: Vector towards the point in consideration</li>
<li>$|x_1|, |x_2|$ are the absolute values of the first and second element of the
vector (the <em>x</em> and <em>y</em> coordinates).</li>
<li>$s$: The size of the square.</li>
</ul>
<p>Using the max function we sort of select to which boundary the distance
will be calculated. Then we substract the size of the smaller square
(highlighted with light blue). The resulting distance is then negative
if the point lies within the square, and positive otherwise.</p>
<h2 id="combining-shapes">Combining shapes</h2>
<p>To make more complex shapes, it is often useful to combine multiple
simple shapes. To do this, we introduce some basic operations on the
returned distances of a simple shape <sup class="footnote-ref" id="fnref:1"><a href="#fn:1">1</a></sup>.</p>
<h3 id="inverse">Inverse</h3>
<p>We made the arbitrary decision to say that the distance is negative when
a point lies within the shape. To get the inverse of a shape we simply
need to negate each distance value.</p>
<p>$$\forall x, y: \neg S(x, y) = -S(x, y)$$</p>
<h3 id="union">Union</h3>
<p>The union of two shapes can be retrieved by using the min function on
both distance functions.</p>
<p>$$ \forall x, y : U(x, y) = \min(S_1(x, y), S_2(x, y)) $$</p>
<p>Remember that the distance value is negative when the point lies in the
shape. The lowest value will win here, so if one of those distances is
negative (the point belongs to at least one shape), it will return the
negative value. Thus combining both shapes to a single one.</p>
<h3 id="difference">Difference</h3>
<p>The difference of two shapes contains all points in $S_1$ excluding the
points in $S_2$. This is defined as follows:</p>
<p>$$\forall x, y : D(x, y) = \max(S_1(x, y), -S_2(x, y))$$</p>
<p>Consider the example where $S_1(x, y)=−2$ and
$S_2(x, y)=−1$. In short, the current point $(x, y)$ belongs to
both $S_1$ and $S_2$. Using the above function for the difference, the value
from $S_2$ gets negated: $−S_2(x, y) = 1$. Due to the max function,
this value will win (it&rsquo;s higher than -2), and therefore, it will <em>not</em>
be part of the new shape. Which is precisely what we want, because we
want all points that are part of $S_1$ but not part of $S_2$.</p>
<h3 id="intersection">Intersection</h3>
<p>The intersection of two shapes contains all points that are both part of
$S_1$ and $S_2$. It is defined as follows:</p>
<p>$$\forall x, y: I(x, y) = \max(S_1(x, y), S_2(x, y))$$</p>
<p>A point will be part of the new shape if and only if both distances are
negative. If one distance is positive, the max function will return this
value, and a positive value means not part of the shape. This results in
a shape which includes only points that are both part of $S_1$ and $S_2$.</p>
<h2 id="opengl-implementation">OpenGL implementation</h2>
<p>So how do we translate these principles to some working code? Let us
first introduce some basic OpenGL concepts before throwing the shader
code at you.</p>
<h3 id="shaders-and-drawing-modes">Shaders and drawing modes</h3>
<p>I will not go too deep in the basics of OpenGL, but a modern OpenGL
pipeline consists of multiple <em>shaders</em>, small programs you can write
yourself. At the very minimum you&rsquo;ll need a <em>vertex shader</em> and a
<em>fragment shader</em>, which determine where the main &ldquo;drawing points&rdquo; will
be positioned and the color of the individual pixels respectively. The
pipeline is illustrated in fig. 3.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/gl-pipeline.png" alt="
OpenGL pipeline illustrated.
" />
<figcaption>
OpenGL pipeline illustrated.
</figcaption>
</figure>
</p>
<p>You can define your own attributes for each vertex, for example the
position, colour, or orientation.</p>
<p>OpenGL has several modes for generating the actual primitives. These are
illustrated in fig. 4.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/gl-primitives.png" alt="
OpenGL primitive generation modes.
" />
<figcaption>
OpenGL primitive generation modes.
</figcaption>
</figure>
</p>
<p>For a more in depth OpenGL introduction, I would recommend <a href="http://www.labri.fr/perso/nrougier/teaching/opengl/" target="_blank">this
tutorial</a>, <a href="http://antongerdelan.net/opengl/" target="_blank">Anton&rsquo;s
OpenGL tutorials</a>, or
<a href="http://opengl-tutorial.org" target="_blank">opengl-tutorial.org</a>.</p>
<p><small>Above pictures courtesy of N. Rougier<sup class="footnote-ref" id="fnref:2"><a href="#fn:2">2</a></sup></small></p>
<h3 id="general-2d-shape-shaders">General 2D shape shaders</h3>
<p>For the 2D shapes we want to draw, we&rsquo;re going to use the points drawing
mode. OpenGL allows you to specify the point size, and the fragment
shader will be called for each pixel in the point.</p>
<p>Let&rsquo;s check the vertex shader were we position our vertices, and
configure the point size.</p>
<h4 id="vertex-shader">Vertex shader</h4>
<p>Listing 1: The vertex shader code for our 2D shapes</p>
<pre><code class="language-glsl">// Uniforms
// ------------------------------------
uniform float antialias;
uniform mat4 ortho;
// Attributes
// ------------------------------------
attribute vec2 position;
attribute float size;
attribute vec4 color;
attribute float rotation;
attribute float linewidth;
// Varyings
// ------------------------------------
varying float v_size;
varying vec4 v_color;
varying vec2 v_rotation;
varying float v_antialias;
varying float v_linewidth;
// Main
// ------------------------------------
void main (void)
{
v_size = size;
v_linewidth = linewidth;
v_antialias = antialias;
v_color = color;
v_rotation = vec2(cos(rotation), sin(rotation));
gl_Position = ortho * vec4(position, 0, 1);
gl_PointSize = M_SQRT2 * size + 2.0 * (linewidth + 1.5*antialias);
}
</code></pre>
<p>We first define some <em>uniforms</em>, <em>attributes</em>, and <em>varyings</em>. Uniforms
are variables which are the same for each vertex. Attributes are
variables defined per vertex, and with varyings we can pass some data to
the next steps in the pipeline (for example, the fragment shader).</p>
<p>Each vertex has a position, and this is where our 2D shape will be
drawn. The matrix <code>ortho</code> is used for the proper projection of the
vertex to your screen. We will not explain this in-depth, but if you
want to know more please refer to <a href="http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/" target="_blank">this
tutorial</a>
on opengl-tutorial.org.</p>
<p>Our shapes are larger than one pixel, so we need to change
<code>gl_PointSize</code>. Our shapes also have a size attached to them, but for
the point size we scale this size with $\sqrt{2}$ (ignore the extra
size for antialias en linewidth for now). We do this because our shapes
can be rotated. To fit a rotated square of size <em>x</em> in another square,
we need a bigger square of size $x\sqrt{2}$ (I hope you guys remember
Pythagoras). This is visualized in fig. 5.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/point-rotation.png" alt="
Rotation of a square
" />
<figcaption>
Rotation of a square
</figcaption>
</figure>
</p>
<p>Furthermore, we pass along some variables to the next steps in the
pipeline (size, line width, antialias, color, rotation). Note we create
a direction vector for the rotation from the given rotation in radians.</p>
<h4 id="fragment-shader">Fragment shader</h4>
<p>Listing 2: The fragment shader code</p>
<pre><code class="language-glsl">// Varyings
// ------------------------------------
varying float v_size;
varying vec4 v_color;
varying vec2 v_rotation;
varying float v_antialias;
varying float v_linewidth;
// Main
// ------------------------------------
void main()
{
vec2 P = gl_PointCoord.xy - vec2(0.5,0.5);
P = vec2(v_rotation.x*P.x - v_rotation.y*P.y,
v_rotation.y*P.x + v_rotation.x*P.y) * v_size;
float size = v_size/M_SQRT2;
float distance = shape_func(P, size);
gl_FragColor = filled(distance, v_linewidth, v_antialias, v_color);
}
</code></pre>
<p>Note we have the same varying definitions as in the vertex shader. These
contain values as passed from the vertex shader. Also note the usage of
the built in variable <code>gl_PointCoord</code>. We specified in the vertex shader
the size of our point in pixels, and for each pixel in this point the
fragment shader gets called. The <code>gl_PointCoord</code> contains the
coordinates <strong>inside the point</strong>. The <em>x</em> and <em>y</em> attributes from
<code>gl_PointCoord</code> range from 0.0 to 1.0, where (0, 0) is the bottom left
corner of the point, and (1, 1) is the top right corner of the point.</p>
<p>There are several operations applied to these coordinates:</p>
<p><strong>Step 1.</strong> First we substract $\begin{bmatrix}0.5 \\ 0.5 \end{bmatrix}$.
This makes sure the origin is in the center of the point, because
the distance functions we defined earlier in this article
require that.</p>
<p><strong>Step 2.</strong> Next we apply a rotation transformation to the point.</p>
<p>Remember the transformation matrix is:</p>
<p>$$ \begin{bmatrix} \text{cos}(\theta) &amp; \text{-sin}(\theta) \\
\text{sin}(\theta) &amp; \text{cos}(\theta) \end{bmatrix}$$</p>
<p>If you look closely at the code you see that the vector <code>P</code> gets
multiplied with this matrix.</p>
<p>$$\begin{bmatrix}nx \\ ny\end{bmatrix} = \begin{bmatrix}x \\ y
\end{bmatrix} \begin{bmatrix} \mathrm{cos}(\theta) &amp; \mathrm{-sin}(\theta) \\
\mathrm{sin}(\theta) &amp; \mathrm{cos}(\theta) \end{bmatrix} = \begin{bmatrix} x
\mathrm{cos}(\theta) - y \mathrm{sin}(\theta) \\ x
\mathrm{sin}(\theta) + y \mathrm{cos}(\theta) \end{bmatrix} $$</p>
<p><strong>Step 3.</strong> We also scale the coordinates with <code>v_size</code>.</p>
<p>These transformations are visualized in fig. 6.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/transformations.png" alt="
Transformation of the coordinates within a point
" />
<figcaption>
Transformation of the coordinates within a point
</figcaption>
</figure>
</p>
<p>The green region in the bottom of fig. 6 is our canvas for drawing our
shape. This is done by <code>shape_func()</code>, any function that returns the
distance to the boundary of a shape as explained earlier in this
article.</p>
<p>The <code>filled()</code> function determines the color for the current pixel
determined by the returned distance of <code>shape_func()</code>. Simply put: if
the returned distance is negative, it returns a color (because it&rsquo;s part
of the shape), otherwise it makes the current pixel transparent. It also
applies some anti-aliasing techniques which I don&rsquo;t know the details of,
so we will not cover this in-depth.</p>
<h2 id="example-curved-arrows">Example: curved arrows</h2>
<p>To conclude this article we will look into the distance function of a
curved arrow head. A curved arrow head can be constructed using the
inverse of the union of three circles. This is visualized in fig. 7, and
the accompanying shader code can be seen in lst. 3.</p>
<p>
<figure >
<img src="https://lucasvandijk.nl/img/blogimages/opengl-points/curved-arrows.png" alt="
Construction of a curved arrow head
" />
<figcaption>
Construction of a curved arrow head
</figcaption>
</figure>
</p>
<p>Listing 3: GLSL function to the distance of an arrow</p>
<pre><code class="language-glsl">/**
* Computes the signed distance to an curved arrow
*
* Parameters:
* -----------
* texcoord : Point to compute distance to
* size : size of the arrow head in pixels
*
* Return:
* -------
* Signed distance to the arrow
*
*/
float arrow_curved(vec2 texcoord, float size)
{
vec2 start = -vec2(size/2, 0.0);
vec2 end = +vec2(size/2, 0.0);
float height = 0.5;
vec2 p1 = start + size*vec2(0, -height);
vec2 p2 = start + size*vec2(0, +height);
vec2 p3 = end;
// Head : 3 circles
vec2 c1 = circle_from_2_points(p1, p3, 6.0*size).zw;
float d1 = length(texcoord - c1) - 6*size;
vec2 c2 = circle_from_2_points(p2, p3, 6.0*size).xy;
float d2 = length(texcoord - c2) - 6*size;
vec2 c3 = circle_from_2_points(p1, p2, 3.0*size).xy;
float d3 = length(texcoord - c3) - 3*size;
return -min(d3, min(d1,d2));
}
</code></pre>
<p>We first define the arrow corner points as <code>p1</code>, <code>p2</code> and <code>p3</code>. We use a
helper function which calculates the center point of a circle through
two points with a given radius <code>r</code>.</p>
<p>The distance to these circles are then easily calculated, and we use the
min function to get the union of these three circles. Our arrow head is
exactly the area <em>not</em> covered by these circles (see fig. 7), and
therefore we return the inverse of this union.</p>
<h3 id="references">References</h3>
<div class="footnotes">
<hr />
<ol>
<li id="fn:1"><p>N. P. Rougier, “Antialiased 2D grid, marker, and arrow shaders,”
<em>Journal of Computer Graphics Techniques (JCGT)</em>, vol. 3, no. 4, pp.
1–52, November 2014 [Online]. Available:
<a href="http://jcgt.org/published/0003/04/01/" target="_blank">http://jcgt.org/published/0003/04/01/</a></p>
<a class="footnote-return" href="#fnref:1"><sup>^</sup></a></li>
<li id="fn:2"><p>“Modern openGL tutorial with python.” [Online]. Available: