Roadmap & Tracker #1

The plan is not definite. Feel free to tweak it!

• Append your ID after an item if you are working on it.
• Check the box and remove the "[IMPORTANT]" and BOLD after finishing to focus on the important todos.

Todos/Contributions

Setup @tongyx361

• Run the first simulation
• Setup the demo app
• Curate the development guide

Features

• Concurrent simulations @tongyx361
  • Asynchronous in Streamlit (nest_asyncio)
  • Separate progress bars
• Better platform configuration
  • Clear available agent action suggestions for each platform (e.g. Twitter, Reddit, Weibo)
• Context length limit
  • Summarization
  • Observation sampling
• [IMPORTANT] Simulation record cache
• [IMPORTANT] Post composing / revision suggestion based on LLMs
  • Convenient suggestion adoption
• [IMPORTANT] Simulation result analysis based on LLMs
• Agent information collection utilities (Only constructing a "small world" in a complex network should be enough, e.g., active users under a specific topic.)
• VLM support

Model-Related

• Evaluation: curate ground-truth data to evaluate the model's performance for simulation (see [Question] Datasets camel-ai/oasis#11 (comment) for reference)
  • Single action: predicting users' reactions to a post
  • Dynamics: comparing the dynamics of the statistics of the propagation graph (e.g. depth, # of nodes) with the ground truth data
• Optimization: Tune the system (e.g. simulation parameters, prompts) to optimize the simulation accuracy
• Tables of evaluation / ablation results in the project report

Analysis & Visualization

• Fill up the "Analysis" section in the demo app.
  • Better to be realtime, which should be feasible by reading from the databases timely.
  • [IMPORTANT] Statistics v.s. Time: see temporal_graph.py for utilities
  • Simulation animation: e.g., how reposts construct a graph as time goes on
  • ...