DL2

Deep Learning Learning Platform, a full-stack platform for personalized learning path discovery, based on RAG, citation graphs and semantic search.

1. MySQL or MariaDB w./ schema dl2, on localhost:3306: systemctl enable --now mysqld
2. Neo4j on localhost:7474: <NEO4J_HOME>/bin/neo4j restart

1. User & paper service on localhost:8080: cd UserService;./gradlew run
2. Neo4j graph service on localhost:8081: cd Neo4jService;./gradlew run
3. Frontend service on localhost:4200: cd frontend;ng serve

1. Vectorized search service w./ weaviate on Docker: sudo docker run -d -p 8090:8090 -p 50051:50051 dl2-vectorized;cd SearchLambda;python serve_test.py
2. Gemini-based GraphRAG: cd DataAnalysis/llm;python graph_rag.py
3. GAT-based paper recommendation: cd DataAnalysis;python gat_pred.py

1. Initialization w./ renowned PapersWithCode methods and tasks: cd CrawlerService;npm run init
2. Expand the graph w./ Neo4j uncharted papers, exported as CSVs: npm run import
3. Daily routine crawling w./ subscription to ArXiV: npm run subscribe

Uses Angular & Angular material to host a frontend. Angular material provides an out-of-the-box development experience considering UI design.

• A seamless guide through deep learning papers nowadays, where you get customized recommendations based on browsing history and papers' citation graph, as well as access to a Markdown driven method introduction with only a few clicks.
• Graph visualisation: Uses D3, a graph engine with embedded physics engine that handles the forces among nodes.
• WebASM integrated LSP: Uses Jedi, Python language metatools to provide code autocompletions, embedded in JS using Pyodide.
• DataComp(beta) where you can upload datasets and compete with each other. The competition footprint can thus enhance our recommendation system.

• Weaviate: Using Weaviate Vector Database API, we can retrieve the vector encoded by HuggingFace Model all-MiniLM-L6-v2, and find the objects closest to the query concepts.
• Neo4j: Neo4j is a graph database based on Cypher, which helps us find the neighbors of a paper on the citation graph with great convenience and performance.

• Recommendation System based on papers' citation graph and text data, and users' interest field.
• A GNN fine-grained field classifier for newly published papers.

• Autoscaler deployment of backend
• WSGI service of RFS
• Cache the python file in the frontend with IndexedDB
• New DataComp Recommendation System
• A ChatBot with OpenAI API

• Kubernetes deployment with autoconfigured docker images
• ElasticSearch deployment with distributed ML searching capabilities
• Prometheus TSDB with advanced monitoring and logging of service statuses