Skip to content

Documentation: document how to pull bookmarks from Firefox Sync #648

Description

@khimaros

Wiki Page URL

https://github.com/ArchiveBox/ArchiveBox/wiki/Quickstart#2-get-your-list-of-urls-to-archive

Suggested Edit

document how to pull bookmarks and/or history from the Firefox Sync server.

this enables the user to simply bookmark pages they want to archive on their desktop and mobile browser and they will automatically be archived by archivebox.

here's what worked for me.

determine your Firefox Sync client id:

$ python3 -m pip install PyFxA

$ fxa-client --bearer -u <YOUR EMAIL> \
    --account-server https://api.accounts.firefox.com/v1 \
    --oauth-server https://oauth.accounts.firefox.com/v1
# ---- BEARER TOKEN INFO ----
# User: <YOUR EMAIL>
# Scopes: profile
# Account: https://api.accounts.firefox.com/v1
# Oauth: https://oauth.accounts.firefox.com/v1
# Client ID: <YOUR CLIENT ID>
# ---------------------------

import all of your bookmarks into archivebox:

$ git clone https://github.com/eNote-GmbH/syncclient

$ cd syncclient

$ python3 -m pip install -e .

$ python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> \
    get_records bookmarks \
        | jq '.[].payload | fromjson | select(.type == "bookmark") | .bmkUri)' \
        | archivebox add

this also works with a self hosted Firefox Sync server by setting the env var TOKENSERVER_URL="https://<YOUR SYNC SERVER>/token". you can configure this to run on a regular schedule eg. via cron.

with a bit of work, you could make this only archive bookmarks with a specific tag or folder.

Activity

  1. kominoshja commented on Nov 30, 2021

    @kominoshja

    Note: This requires disabling 2FA while setting it up

  2. kominoshja commented on Nov 30, 2021

    @kominoshja

    When you issue python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_records bookmarks, you get something like

    "6J***********3Vk"
    "3_***********Iw7"
    "U1***********ah"
    

    therefore, jq fails. You'd need to run python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_record bookmarks --decrypt 6J***********3Vk | jq '.payload | fromjson | select(.type == "bookmark") | .bmkUri'

    I created the following script to iterate over the response and add bookmarks to archivebox (Keep in mind you need to change BATCH_SIZE to archive more than 4 at a time): https://gist.github.com/kominoshja/484dd83f978079daf6c3bacebb21fe37

  3. pirate commented on Jan 19, 2024

    @pirate
    Member

    Going to close this for now, I'll add a link to this issue from the README in case it helps ppl in the future trying to set this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions