Help:Toolforge

Toolforge

• Cloud Services overview
• Toolforge user docs
• Toolforge changelog

Get started

• Quickstart: set up and get access
• How Toolforge works
• Rules you must follow
• Tutorials

Build and run tools

• Navigate tool accounts and files
• Build container images for tools
• Run a web service
• Schedule and manage jobs
• Manage tool runtime configuration (envvars)
• Language-specific details:
  • Python
  • Pywikibot
  • Node.js
  • PHP
  • ...more languages/frameworks
• Use Redis for caching
• Index content with Elasticsearch

Access shared storage and databases

• Access shared storage and public wiki dumps
• Access the Wiki Replicas databases
• Access replica search indices
• Manage tool databases

Share and maintain tools

• Set up version control and code review
• Develop successful tools
• Find and share tools on Toolhub
• Delete a tool
• Contribute to Toolforge

Get help

• How and where to get help
• Troubleshooting

Useful links

• Toolforge admin docs
• List of tools
• Toolforge Admin Console (toolsadmin)

[]

This is the landing page for Toolforge user documentation. It provides an overview of Toolforge features and capabilities, then directs you to where you can learn more or get started using the platform.

Create or deploy your own tools on Toolforge

Toolforge quickstart Create and manage tool accounts

What is Toolforge?

Toolforge is a free cloud hosting platform for Wikimedia contributors. It provides web servers, data access, job management, and other features to help developers maintain tools and bots.

Toolforge is part of the Wikimedia Cloud Services (WMCS) suite of services. It is supported by Wikimedia Foundation staff and volunteers.

To learn more about how Toolforge works, see About Toolforge.

What are tools and bots?

Tools are software applications that help people working on Wikimedia projects. Bots are automated programs that help wiki projects by doing crucial but repetitive tasks. Tools and bots can do all kinds of tasks, like help editors discover content to improve, simplify edit-a-thon planning, visualize wiki content, help fight vandalism, and much more.

Many tools and bots use MediaWiki APIs to access wiki content, metadata, analytics data, and more. You can use these APIs to build apps that contribute content or new functionality to the Wikimedia ecosystem, then host your app on Toolforge.

See the Wikimedia technical ecosystem overview to learn more about how tools and bots help the movement.

Main features of Toolforge

• Free, reliable, and scalable cloud hosting for:
  • Web services using PHP, Python, Node.js, Java, and other languages or frameworks
  • Continuous bots and scheduled tasks
• Kubernetes backend for orchestration
• Access to replicas of production wiki databases, and other data services
• Access to shared storage, which includes public dumps of Wikimedia content and data
• Shared management of tool accounts (where tools and bots are stored)
• Support for mosh, SSH, SFTP
• Version control via Git
• Options for tools to send and receive email
• Support for Redis
• Support for Elasticsearch

Constraints of Toolforge

• All code run on Toolforge must benefit the Wikimedia movement.
• Use of Toolforge is subject to the general Cloud Services Terms of Use and the Toolforge-specific rules and policies.
  • The Toolforge rules include publishing your code under an OSI approved open source license.
• Because Toolforge is a managed service, you are limited to the packages provided by the Wikimedia Foundation on its servers or installed through build service. If your project requires that you administer your own virtual server, consider Cloud VPS instead.
• Memory and storage is limited to 2 virtual CPUs and 8 GB of RAM. The maximum recommended per-job memory limit is 4GB. If your project requires more memory, you can request a quota increase, or use Cloud VPS instead. Learn more about quotas.
• Make sure that your tools don't open too many connections to the wiki sites in parallel, currently there's a hard limit of 500 simultaneous connections to each wiki so any misbehaving tool might be stopped without warning if they are using more than 50 simultaneous connections to wikis at a time.

What tools are hosted on Toolforge?

Toolforge hosts various types of tools, including web services, continuous bots, and scheduled tasks.

To explore tools by type, visit Toolhub. To see all the tools currently hosted on Toolforge, use the Toolforge admin console ("toolsadmin").

Before you start

To use Toolforge, you need:

• Some programming knowledge
• An understanding of Unix command line
• An SSH client (included with most modern operating systems)
• A Wikimedia developer account, and a Wikimedia account (used to log in to Wikipedia and other Wikimedia wikis)

Get started

Follow the Toolforge quickstart guide to create your tool account and get started using Toolforge.

To get familiar with MediaWiki APIs, try a tutorial, or prototype your code in PAWS, a Jupyter notebook installation hosted by Wikimedia.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support

• Chat in real time in the IRC channel #wikimedia-cloud ^connect or the bridged Telegram group
• Discuss via email after you have subscribed to the cloud@ mailing list

Stay aware of critical changes and plans

• Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
• Read the News wiki page

Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)