Mootlib

A lightweight Python library for semantic search across prediction markets. Think metaforecast but simpler and Python-native.

• Cross-platform forecast aggregation
• Semantic search powered by BAAI/bge-m3 embeddings
• Automatic data caching and management
• Direct market data access

Mootlib fetches encrypted market data and embeddings from GitHub artifacts, updated hourly via Actions. You'll need your own encryption key to access the data.

1. Fork and install the repo
2. Generate your encryption key:

   from cryptography.fernet import Fernet
   key = Fernet.generate_key()

3. Add the key as MOOTLIB_ENCRYPTION_KEY in your repo secrets
4. Enable and run the GitHub Actions workflow
5. Start querying markets with mootlib

• Pool forecasts across multiple prediction market platforms using
• semantic similarity search with BAAI/bge-m3-based embeddings
• Automatic caching and data management
• Direct access to market data and embeddings

No fancy database behind the scenes, mootlib is just a Python library fetching some data from a github repo. The GH Actions workflow is used to fetch the data, compute embeddings, and release an artefact every hour or so; the code fetches the latest artefacts. We encrypt the artefacts using the key MOOTLIB_ENCRYPTION_KEY to keep your own parsing end embedding private. This means that you will need to set up your repo and your GH Actions workflow before using the library.

1. Fork the repo and install it
2. Create the MOOTLIB_ENCRYPTION_KEY using the Fernet.generate_key() function from cryptography.fernet and save it (see below)
3. Add the MOOTLIB_ENCRYPTION_KEY to the repo secrets
4. Ensure the GH Actions workflow is enabled and has run
5. Enjoy querying the data from mootlib

• Add a database?
• Improve prediction quality metrics
• Small LLM-based filter for question relevance
• More integrations:
  • Kalshi
  • https://www.randforecastinginitiative.org
  • SMarkets

pip install mootlib

The library requires several environment variables to function:

• MOOTLIB_ENCRYPTION_KEY: Required for decrypting market data. You have to create your own using the Fernet.generate_key() function from cryptography.fernet.
• DEEPINFRA_TOKEN: Required for computing embeddings. You can get it by going to https://deepinfra.com/ and creating an account.
• GJO_EMAIL and GJO_PASSWORD: Optional, for Good Judgment Open access. You can get it by going to https://goodjudgment.io/ and creating an account.

Once you have all your keys, you can set them up in two ways:

Create a .env file in your project root:

MOOTLIB_ENCRYPTION_KEY="your-key-here"
DEEPINFRA_TOKEN="your-token-here"
GJO_EMAIL="your-email@example.com"  # Optional
GJO_PASSWORD="your-password"        # Optional

Then in your Python code:

from dotenv import load_dotenv
load_dotenv()  # Load environment variables from .env

from mootlib import MootlibMatcher
matcher = MootlibMatcher()

# Unix/macOS
export MOOTLIB_ENCRYPTION_KEY="your-key-here"
export DEEPINFRA_TOKEN="your-token-here"

# Windows PowerShell
$env:MOOTLIB_ENCRYPTION_KEY="your-key-here"
$env:DEEPINFRA_TOKEN="your-token-here"

Add these secrets in your repository's Settings → Secrets and Variables → Actions:

• MOOTLIB_ENCRYPTION_KEY
• DEEPINFRA_TOKEN
• GJO_EMAIL (optional)
• GJO_PASSWORD (optional)

Then use them in your workflow:

env:
  MOOTLIB_ENCRYPTION_KEY: ${{ secrets.MOOTLIB_ENCRYPTION_KEY }}
  DEEPINFRA_TOKEN: ${{ secrets.DEEPINFRA_TOKEN }}

from mootlib import MootlibMatcher

# Initialize the matcher
matcher = MootlibMatcher()

# Find similar questions
similar = matcher.find_similar_questions(
    "Will Russia invade Moldova in 2024?",
    n_results=3,
    min_similarity=0.7
)

# Print the results
for question in similar:
    print(f"\n{question}")

The main interface for finding similar questions across prediction markets.

matcher = MootlibMatcher(cache_duration_minutes=30)

Parameters:

• cache_duration_minutes: How long to keep downloaded data in cache (default: 30)

Access the raw markets DataFrame containing all prediction market data:

markets_df = matcher.markets_df

The DataFrame contains columns:

• question: The market question text
• source_platform: Platform where the market is from
• formatted_outcomes: Current probabilities/outcomes
• url: Link to the original market
• n_forecasters: Number of forecasters
• volume: Trading volume/liquidity
• published_at: Publication datetime

Access the embeddings DataFrame containing question vectors:

embeddings_df = matcher.embeddings_df

The DataFrame contains columns:

• text: The question text
• embedding: The numerical embedding vector

Note: Embeddings are computed on-demand and cached for future use.

similar = matcher.find_similar_questions(
    query="Will Tesla stock reach $300 in 2024?",
    n_results=5,
    min_similarity=0.5
)

Parameters:

• query: The question to find similar matches for
• n_results: Number of similar questions to return (default: 5)
• min_similarity: Minimum similarity score 0-1 (default: 0.5)

Returns a list of SimilarQuestion objects with the following attributes:

• question: The text of the prediction market question
• similarity_score: How similar this question is to the query (0-1)
• source_platform: The platform where this question was found
• formatted_outcomes: String representation of possible outcomes and probabilities
• url: URL to the original market (optional)
• n_forecasters: Number of people who made predictions (optional)
• volume: Trading volume or liquidity (optional)
• published_at: When the market was published (optional)

from mootlib import MootlibMatcher

matcher = MootlibMatcher()

# Search for AI-related questions
ai_questions = matcher.find_similar_questions(
    "Will AGI be achieved by 2025?",
    n_results=3,
    min_similarity=0.7
)

# Search for geopolitical questions
geo_questions = matcher.find_similar_questions(
    "Will China invade Taiwan in 2024?",
    n_results=3,
    min_similarity=0.7
)

# Print results
for q in ai_questions + geo_questions:
    print(f"\n{q}\n{'=' * 80}")

from mootlib import MootlibMatcher

matcher = MootlibMatcher()

# Find similar questions and access their details
similar = matcher.find_similar_questions("Will SpaceX reach Mars by 2025?")

for q in similar:
    print(f"\nQuestion: {q.question}")
    print(f"Platform: {q.source_platform}")
    print(f"Current Probabilities: {q.formatted_outcomes}")
    if q.url:
        print(f"Market URL: {q.url}")
    if q.n_forecasters:
        print(f"Number of Forecasters: {q.n_forecasters}")
    print("-" * 80)

from mootlib import MootlibMatcher

matcher = MootlibMatcher()

# Get all market data
markets_df = matcher.markets_df
print(f"Total markets: {len(markets_df)}")
print("\nMarkets by platform:")
print(markets_df["source_platform"].value_counts())

# Get question embeddings
embeddings_df = matcher.embeddings_df
print(f"\nTotal questions with embeddings: {len(embeddings_df)}")

# Filter markets by platform
manifold_markets = markets_df[markets_df["source_platform"] == "Manifold"]
print(f"\nManifold markets: {len(manifold_markets)}")

# Get high-volume markets
high_volume = markets_df[markets_df["volume"] > 1000]
print(f"\nHigh volume markets: {len(high_volume)}")

1. Clone the repository

git clone https://github.com/vigji/mootlib.git
cd mootlib

2. Install dependencies with uv

pip install uv
uv venv
source .venv/bin/activate  # On Unix/macOS
# or
.venv\Scripts\activate     # On Windows
uv pip install -e ".[dev]"

We use Ruff for all Python linting and formatting:

# Format code
ruff format .

# Run linter
ruff check .

# Run linter with automatic fixes
ruff check --fix .

We use Git tags for versioning. The version number is automatically derived from the latest tag using hatch-vcs.

To create a new release, you have two options:

1. Quick Release (via Git tag):

# Create and push a new version tag (e.g., v0.1.1)
git tag -a v0.1.1 -m "Description of changes"
git push origin v0.1.1

This will automatically trigger the release workflow.

2. Full Release (via GitHub UI):
   • Create and push a tag as above
   • Go to GitHub -> Releases -> Create a new release
   • Choose the tag you just pushed
   • Add detailed release notes
   • Click "Publish release"

In both cases, the release workflow will automatically:

• Run all tests
• If tests pass, build the package
• Publish to PyPI using trusted publishing

Note: Using the GitHub UI method allows you to add more detailed release notes and attachments, but both methods will publish to PyPI.

We use pre-commit hooks to ensure code quality. Install them with:

pre-commit install

This will automatically run Ruff and other checks before each commit.

• Maximum line length: 88 characters (enforced by Ruff)
• Use pathlib over os.path
• Use functions only where you see opportunity for code reuse
• Use classes sparingly and when it makes sense over functions
• Use loops to streamline operations repeated more than once
• Document briefly middle-length functions, fully annotate only complex ones

pytest

mypy mootlib tests

Contributions are welcome! Please feel free to submit a Pull Request.

This project is licensed under the MIT License - see the LICENSE file for details.