From ed7b4c56e73c520069ea1f9c641d45137a120d3c Mon Sep 17 00:00:00 2001 From: Kenneth Belitzky Date: Thu, 9 Apr 2026 23:38:20 -0300 Subject: [PATCH] Add structkit to CLI Tools > Productivity Tools structkit is a YAML-first project scaffolding tool with remote content fetching (GitHub, S3, GCS, HTTP) and MCP integration for AI-native workflows. Listed alongside cookiecutter and copier. --- README.md | 1012 ++++++++++++++++++++++++++--------------------------- 1 file changed, 488 insertions(+), 524 deletions(-) diff --git a/README.md b/README.md index 89a4a6d..c825eda 100644 --- a/README.md +++ b/README.md @@ -83,41 +83,47 @@ An opinionated list of Python frameworks, libraries, tools, and resources. - [Text Processing](#text-processing) - [HTML Manipulation](#html-manipulation) -- [File Format Processing](#file-format-processing) -- [File Manipulation](#file-manipulation) +- [PDF](#pdf) +- [Office Documents](#office-documents) +- [Markdown](#markdown) -**Media** +**Multimedia** +- [Audio](#audio) - [Image Processing](#image-processing) -- [Audio & Video Processing](#audio--video-processing) -- [Game Development](#game-development) +- [Video](#video) -**Python Language** - -- [Implementations](#implementations) -- [Built-in Classes Enhancement](#built-in-classes-enhancement) -- [Functional Programming](#functional-programming) -- [Asynchronous Programming](#asynchronous-programming) -- [Date and Time](#date-and-time) - -**Python Toolchain** - -- [Environment Management](#environment-management) -- [Package Management](#package-management) -- [Package Repositories](#package-repositories) -- [Distribution](#distribution) -- [Configuration Files](#configuration-files) - -**Security** +**Cryptography & Security** - [Cryptography](#cryptography) -- [Penetration Testing](#penetration-testing) +- [Security](#security) -**Miscellaneous** +**OS & System** +- [OS](#os) +- [Files](#files) +- [Date and Time](#date-and-time) +- [Internationalization](#internationalization) +- [Compatibility](#compatibility) +- [Event](#event) +- [Process](#process) +- [Concurrency and Parallelism](#concurrency-and-parallelism) + +**Other** + +- [Computer Vision](#computer-vision) +- [Configuration Files](#configuration-files) +- [Design Patterns](#design-patterns) +- [Functional Programming](#functional-programming) +- [Game Development](#game-development) +- [Geolocation](#geolocation) - [Hardware](#hardware) -- [Microsoft Windows](#microsoft-windows) -- [Miscellaneous](#miscellaneous) +- [Package Management](#package-management) +- [Package Repositories](#package-repositories) +- [Robotics](#robotics) +- [RPC Servers](#rpc-servers) +- [Websocket](#websocket) +- [WSGI Servers](#wsgi-servers) --- @@ -125,77 +131,83 @@ An opinionated list of Python frameworks, libraries, tools, and resources. ## AI and Agents -_Libraries for building AI applications, LLM integrations, and autonomous agents._ +_Frameworks and tools for building AI agents._ -- Agent Skills - - [django-ai-plugins](https://github.com/vintasoftware/django-ai-plugins) - Django backend agent skills for Django, DRF, Celery, and Django-specific code review. - - [sentry-skills](https://github.com/getsentry/skills) - Python-focused engineering skills for code review, debugging, and backend workflows. - - [trailofbits-skills](https://github.com/trailofbits/skills) - Python-friendly security skills for auditing, testing, and safer backend development. -- Orchestration - - [autogen](https://github.com/microsoft/autogen) - A programming framework for building agentic AI applications. - - [crewai](https://github.com/crewAIInc/crewAI) - A framework for orchestrating role-playing autonomous AI agents for collaborative task solving. - - [dspy](https://github.com/stanfordnlp/dspy) - A framework for programming, not prompting, language models. - - [hermes-agent](https://github.com/nousresearch/hermes-agent) - An adaptive AI agent framework that grows with you. - - [langchain](https://github.com/langchain-ai/langchain) - Building applications with LLMs through composability. - - [pydantic-ai](https://github.com/pydantic/pydantic-ai) - A Python agent framework for building generative AI applications with structured schemas. - - [TradingAgents](https://github.com/TauricResearch/TradingAgents) - A multi-agents LLM financial trading framework. -- Data Layer - - [instructor](https://github.com/567-labs/instructor) - A library for extracting structured data from LLMs, powered by Pydantic. - - [llama-index](https://github.com/run-llama/llama_index) - A data framework for your LLM application. - - [mem0](https://github.com/mem0ai/mem0) - An intelligent memory layer for AI agents enabling personalized interactions. -- Pre-trained Models and Inference - - [diffusers](https://github.com/huggingface/diffusers) - A library that provides pre-trained diffusion models for generating and editing images, audio, and video. - - [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models. - - [transformers](https://github.com/huggingface/transformers) - A framework that lets you easily use pre-trained transformer models for NLP, vision, and audio tasks. - - [unsloth](https://github.com/unslothai/unsloth) - A library for faster LLM fine-tuning and training with reduced memory usage. - - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. +- [agno](https://github.com/agno-agi/agno) - A lightweight library for building multi-modal Agents. +- [autogen](https://github.com/microsoft/autogen) - A framework for building multi-agent conversational systems. +- [browser-use](https://github.com/browser-use/browser-use) - Make websites accessible for AI agents with easy browser automation. +- [camel](https://github.com/camel-ai/camel) - A communicative agents framework for LLM role-playing scenarios. +- [crewai](https://github.com/joaomdmoura/crewAI) - Framework for orchestrating role-playing, autonomous AI agents. +- [dspy](https://github.com/stanfordnlp/dspy) - The framework for programming—not prompting—foundation models. +- [haystack](https://github.com/deepset-ai/haystack) - LLM orchestration framework to build customizable, production-ready LLM applications. +- [langchain](https://github.com/langchain-ai/langchain) - Building applications with LLMs through composability. +- [litellm](https://github.com/BerriAI/litellm) - A library for calling Anthropic, Azure, Huggingface, Replicate API. +- [lollms-webui](https://github.com/ParisNeo/lollms-webui) - Lord of Large Language Multimodal Systems Web User Interface. +- [mem0](https://github.com/mem0ai/mem0) - The memory layer for AI agents. +- [metagpt](https://github.com/geekan/MetaGPT) - The multi-agent framework assigns different roles to GPTs to form a collaborative software entity. +- [modelscope-agent](https://github.com/modelscope/modelscope-agent) - An agent framework connecting models in ModelScope with the world. +- [openai-agents-sdk](https://github.com/openai/openai-agents-python) - A lightweight, powerful framework for multi-agent workflows. +- [pydantic-ai](https://github.com/pydantic/pydantic-ai) - Agent framework built on top of Pydantic for production-grade Generative AI applications. +- [semantic-kernel](https://github.com/microsoft/semantic-kernel) - Integrate cutting-edge LLM technology quickly and easily. +- [smolagents](https://github.com/huggingface/smolagents) - An extremely simple library that unleashes language models to tackle complex tasks. +- [superagi](https://github.com/TransformerOptimus/SuperAGI) - A dev-first open source autonomous AI agent framework. ## Deep Learning -_Frameworks for Neural Networks and Deep Learning. Also see [awesome-deep-learning](https://github.com/ChristosChristofidis/awesome-deep-learning)._ +_Frameworks for neural networks and deep learning._ -- [jax](https://github.com/jax-ml/jax) - A library for high-performance numerical computing with automatic differentiation and JIT compilation. -- [keras](https://github.com/keras-team/keras) - A high-level deep learning library with support for JAX, TensorFlow, and PyTorch backends. -- [pytorch-lightning](https://github.com/Lightning-AI/pytorch-lightning) - Deep learning framework to train, deploy, and ship AI products Lightning fast. +- [caffe](https://github.com/BVLC/caffe) - A fast open framework made with expression, speed and modularity in mind. +- [keras](https://github.com/keras-team/keras) - A high-level neural networks library and capable of running on top of either TensorFlow or Theano. - [pytorch](https://github.com/pytorch/pytorch) - Tensors and Dynamic neural networks in Python with strong GPU acceleration. -- [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) - PyTorch implementations of Stable Baselines (deep) reinforcement learning algorithms. + - [ignite](https://github.com/pytorch/ignite) - High-level library to help with training neural networks in PyTorch. - [tensorflow](https://github.com/tensorflow/tensorflow) - The most popular Deep Learning framework created by Google. + - [tensorboard](https://github.com/tensorflow/tensorboard) - TensorFlow's Visualization Toolkit. +- [theano](https://github.com/Theano/Theano) - A library for fast numerical computation. ## Machine Learning -_Libraries for Machine Learning. Also see [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning#python)._ +_Libraries for Machine Learning._ -- [catboost](https://github.com/catboost/catboost) - A fast, scalable, high performance gradient boosting on decision trees library. -- [feature_engine](https://github.com/feature-engine/feature_engine) - sklearn compatible API with the widest toolset for feature engineering and selection. +- [gym](https://github.com/openai/gym) - A toolkit for developing and comparing reinforcement learning algorithms. - [h2o](https://github.com/h2oai/h2o-3) - Open Source Fast Scalable Machine Learning Platform. -- [lightgbm](https://github.com/lightgbm-org/LightGBM) - A fast, distributed, high performance gradient boosting framework. -- [mindsdb](https://github.com/mindsdb/mindsdb) - MindsDB is an open source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning models using standard queries. -- [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for probabilistic graphical models and Bayesian networks. -- [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning with extensive documentation and community support. -- [spark.ml](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/)'s scalable [Machine Learning library](https://spark.apache.org/docs/latest/ml-guide.html) for distributed computing. -- [TabGAN](https://github.com/Diyago/Tabular-data-generation) - Synthetic tabular data generation using GANs, Diffusion Models, and LLMs. +- [mindsdb](https://github.com/mindsdb/mindsdb) - MindsDB is an open source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning models using SQL queries. +- [nni](https://github.com/microsoft/nni) - An open source AutoML toolkit for automate machine learning lifecycle. +- [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning. +- [spark](https://github.com/apache/spark) - Apache Spark Python API. +- [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. - [xgboost](https://github.com/dmlc/xgboost) - A scalable, portable, and distributed gradient boosting library. ## Natural Language Processing _Libraries for working with human languages._ -- General - - [gensim](https://github.com/piskvorky/gensim) - Topic Modeling for Humans. +- Text Processing + - [flair](https://github.com/flairNLP/flair) - Very simple framework for state-of-the-art NLP. + - [gensim](https://github.com/RaRe-Technologies/gensim) - Topic Modelling for humans. - [nltk](https://github.com/nltk/nltk) - A leading platform for building Python programs to work with human language data. - - [spacy](https://github.com/explosion/spaCy) - A library for industrial-strength natural language processing in Python and Cython. + - [pattern](https://github.com/clips/pattern) - A web mining module for the Python. + - [polyglot](https://github.com/aboSamoor/polyglot) - Natural language pipeline supporting hundreds of languages. + - [spacy](https://github.com/explosion/spaCy) - Industrial-strength Natural Language Processing (NLP) with Python and Cython. + - [stanford-corenlp](https://github.com/Lynten/stanford-corenlp) - A Python wrapper for Stanford CoreNLP. - [stanza](https://github.com/stanfordnlp/stanza) - The Stanford NLP Group's official Python library, supporting 60+ languages. -- Chinese - - [funnlp](https://github.com/fighting41love/funNLP) - A collection of tools and datasets for Chinese NLP. + - [textdistance](https://github.com/orsinium-labs/textdistance) - Compute distance between sequences with 30+ algorithms. + - [texthero](https://github.com/jbesomi/texthero) - Text preprocessing, representation and visualization. + - [textblob](https://github.com/sloria/TextBlob) - Providing a consistent API for diving into common natural language processing (NLP) tasks. +- Chinese Text Processing + - [jieba](https://github.com/fxsjy/jieba) - The most popular Chinese text segmentation library. + - [pkuseg-python](https://github.com/lancopku/pkuseg-python) - A toolkit for Chinese word segmentation in various domains. + - [snownlp](https://github.com/isnowfy/snownlp) - A library for processing Chinese text. + - [funNLP](https://github.com/fighting41love/funNLP) - A collection of NLP tools and datasets. ## Computer Vision _Libraries for Computer Vision._ -- [easyocr](https://github.com/JaidedAI/EasyOCR) - Ready-to-use OCR with 40+ languages supported. +- [easyocr](https://github.com/JaidedAI/EasyOCR) - Ready-to-use OCR with 80+ supported languages. - [kornia](https://github.com/kornia/kornia/) - Open Source Differentiable Computer Vision Library for PyTorch. - [opencv](https://github.com/opencv/opencv-python) - Open Source Computer Vision Library. -- [pytesseract](https://github.com/madmaze/pytesseract) - A wrapper for [Google Tesseract OCR](https://github.com/tesseract-ocr). +- [pytesseract](https://github.com/madmaze/pytesseract) - A wrapper for Google Tesseract-OCR. +- [supervision](https://github.com/roboflow/supervision) - Reusable computer vision tools. ## Recommender Systems @@ -203,98 +215,87 @@ _Libraries for building recommender systems._ - [annoy](https://github.com/spotify/annoy) - Approximate Nearest Neighbors in C++/Python optimized for memory usage. - [implicit](https://github.com/benfred/implicit) - A fast Python implementation of collaborative filtering for implicit datasets. -- [scikit-surprise](https://github.com/NicolasHug/Surprise) - A scikit for building and analyzing recommender systems. +- [libffm](https://github.com/guestwalk/libffm) - A library for Field-aware Factorization Machine (FFM). +- [lightfm](https://github.com/lyst/lightfm) - A Python implementation of a number of popular recommendation algorithms. +- [spotlight](https://github.com/maciejkula/spotlight) - Deep recommender models using PyTorch. +- [surprise](https://github.com/NicolasHug/Surprise) - A Python scikit for building and analyzing recommender systems. +- [tensorrec](https://github.com/jfkirk/tensorrec) - A Recommendation Engine Framework in TensorFlow. **Web Development** ## Web Frameworks -_Traditional full stack web frameworks. Also see [Web APIs](#web-apis)._ +_Traditional full stack web frameworks. Also see [RESTful API](#restful-api)._ - Synchronous - - [bottle](https://github.com/bottlepy/bottle) - A fast and simple micro-framework distributed as a single file with no dependencies. - [django](https://github.com/django/django) - The most popular web framework in Python. - - [awesome-django](https://github.com/shahraizali/awesome-django) + - [awesome-django](https://github.com/wsvincent/awesome-django) - [flask](https://github.com/pallets/flask) - A microframework for Python. - [awesome-flask](https://github.com/humiaozuzu/awesome-flask) - [pyramid](https://github.com/Pylons/pyramid) - A small, fast, down-to-earth, open source Python web framework. - [awesome-pyramid](https://github.com/uralbash/awesome-pyramid) - - [fasthtml](https://github.com/AnswerDotAI/fasthtml) - The fastest way to create an HTML app. - - [awesome-fasthtml](https://github.com/amosgyamfi/awesome-fasthtml) - [masonite](https://github.com/MasoniteFramework/masonite) - The modern and developer centric Python web framework. - Asynchronous - - [litestar](https://github.com/litestar-org/litestar) - Production-ready, capable and extensible ASGI Web framework. - - [microdot](https://github.com/miguelgrinberg/microdot) - The impossibly small web framework for Python and MicroPython. - - [reflex](https://github.com/reflex-dev/reflex) - A framework for building reactive, full-stack web applications entirely with Python. - - [robyn](https://github.com/sparckles/Robyn) - A high-performance async Python web framework with a Rust runtime. - - [starlette](https://github.com/Kludex/starlette) - A lightweight ASGI framework and toolkit for building high-performance async services. - - [tornado](https://github.com/tornadoweb/tornado) - A web framework and asynchronous networking library. + - [sanic](https://github.com/sanic-org/sanic) - A Python 3.6+ web server and web framework that's written to go fast. + - [tornado](https://github.com/tornadoweb/tornado) - A Python web framework and asynchronous networking library. + - [fastapi](https://github.com/fastapi/fastapi) - A modern, fast, web framework for building APIs with Python 3.6+ based on standard Python type hints. ## Web APIs -_Libraries for building RESTful and GraphQL APIs._ +_Libraries for building web APIs._ -- Django - - [django-ninja](https://github.com/vitalik/django-ninja) - Fast, Django REST framework based on type hints and Pydantic. - - [django-rest-framework](https://github.com/encode/django-rest-framework) - A powerful and flexible toolkit to build web APIs. - - [strawberry-django](https://github.com/strawberry-graphql/strawberry-django) - Strawberry GraphQL integration with Django. -- Flask - - [apiflask](https://github.com/apiflask/apiflask) - A lightweight Python web API framework based on Flask and Marshmallow. -- Framework Agnostic - - [connexion](https://github.com/spec-first/connexion) - A spec-first framework that automatically handles requests based on your OpenAPI specification. - - [falcon](https://github.com/falconry/falcon) - A high-performance framework for building cloud APIs and web app backends. - - [fastapi](https://github.com/fastapi/fastapi) - A modern, fast, web framework for building APIs with standard Python type hints. - - [sanic](https://github.com/sanic-org/sanic) - A Python 3.6+ web server and web framework that's written to go fast. - - [strawberry](https://github.com/strawberry-graphql/strawberry) - A GraphQL library that leverages Python type annotations for schema definition. - - [webargs](https://github.com/marshmallow-code/webargs) - A friendly library for parsing HTTP request arguments with built-in support for popular web frameworks. +- [django-rest-framework](https://github.com/encode/django-rest-framework) - A powerful and flexible toolkit to build web APIs. +- [eve](https://github.com/pyeve/eve) - REST API framework powered by Flask, MongoDB and good intentions. +- [falcon](https://github.com/falconry/falcon) - A high-performance Python framework for building cloud APIs and web app backends. +- [fastapi](https://github.com/fastapi/fastapi) - A modern, fast (high-performance), web framework for building APIs. +- [flask-restful](https://github.com/flask-restful/flask-restful) - Quickly building REST APIs for Flask. +- [hug](https://github.com/hugapi/hug) - A Python 3 framework for cleanly exposing APIs. +- [sandman2](https://github.com/jeffknupp/sandman2) - Automated REST APIs for existing database-driven systems. +- [connexion](https://github.com/zalando/connexion) - Swagger/OpenAPI First framework for Python on top of Flask with automatic endpoint validation. +- [ninja](https://github.com/vitalik/django-ninja) - Fast, async-ready, OpenAPI-compliant framework for building APIs with Django and Python type hints. ## Web Servers -_ASGI and WSGI compatible web servers._ +_HTTP servers._ -- ASGI - - [daphne](https://github.com/django/daphne) - An HTTP, HTTP/2 and WebSocket protocol server for ASGI and ASGI-HTTP. - - [granian](https://github.com/emmett-framework/granian) - A Rust HTTP server for Python applications built on top of Hyper and Tokio, supporting WSGI/ASGI/RSGI. - - [hypercorn](https://github.com/pgjones/hypercorn) - An ASGI and WSGI Server based on Hyper libraries and inspired by Gunicorn. - - [uvicorn](https://github.com/Kludex/uvicorn) - A lightning-fast ASGI server implementation, using uvloop and httptools. -- WSGI - - [gunicorn](https://github.com/benoitc/gunicorn) - Pre-forked, ported from Ruby's Unicorn project. - - [uwsgi](https://github.com/unbit/uwsgi) - A project aims at developing a full stack for building hosting services, written in C. - - [waitress](https://github.com/Pylons/waitress) - Multi-threaded, powers Pyramid. -- RPC - - [grpcio](https://github.com/grpc/grpc) - HTTP/2-based RPC framework with Python bindings, built by Google. - - [rpyc](https://github.com/tomerfiliba-org/rpyc) (Remote Python Call) - A transparent and symmetric RPC library for Python. +- [gunicorn](https://github.com/benoitc/gunicorn) - Pre-forked, porting to Gevent and others. +- [uvicorn](https://github.com/encode/uvicorn) - A lightning-fast ASGI server. +- [waitress](https://github.com/Pylons/waitress) - Multi-threaded, powers Pyramid. ## WebSocket _Libraries for working with WebSocket._ -- [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and [asyncio](https://docs.python.org/3/library/asyncio.html). -- [channels](https://github.com/django/channels) - Developer-friendly asynchrony for Django. -- [flask-socketio](https://github.com/miguelgrinberg/Flask-SocketIO) - Socket.IO integration for Flask applications. -- [picows](https://github.com/tarasko/picows) - Fastest WebSocket clients and servers with a frame level interface for the most demanding use-cases. -- [websockets](https://github.com/python-websockets/websockets) - A library for building WebSocket servers and clients with a focus on correctness and simplicity. +- [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and asyncio. +- [channels](https://github.com/django/channels) - Developer-friendly asynchronous support for Django. +- [websockets](https://github.com/aaugustin/websockets) - A library for building WebSocket servers and clients focusing on correctness and simplicity. ## Template Engines _Libraries and tools for templating and lexing._ -- [jinja](https://github.com/pallets/jinja) - A modern and designer friendly templating language. +- [jinja2](https://github.com/pallets/jinja) - A modern and designer friendly templating language. - [mako](https://github.com/sqlalchemy/mako) - Hyperfast and lightweight templating for the Python platform. +- [marko](https://github.com/frostming/marko) - A markdown parser with high extensibility. ## Web Asset Management _Tools for managing, compressing and minifying website assets._ - [django-compressor](https://github.com/django-compressor/django-compressor) - Compresses linked and inline JavaScript or CSS into a single cached file. -- [django-storages](https://github.com/jschneier/django-storages) - A collection of custom storage back ends for Django. +- [django-pipeline](https://github.com/jazzband/django-pipeline) - An asset packaging library for Django. +- [django-storages](https://github.com/jschneier/django-storages) - A collection of custom storage backends for Django. +- [fanstatic](https://github.com/zopefoundation/fanstatic) - Packages, optimizes, and serves static file dependencies as Python packages. +- [fileconveyor](https://github.com/wimleers/fileconveyor) - A daemon to detect and sync files to CDNs, S3 and FTP. +- [flask-assets](https://github.com/miracle2k/flask-assets) - Helps you integrate webassets into your Flask app. +- [webassets](https://github.com/miracle2k/webassets) - Bundles, optimizes, and manages unique cache-busting URLs for static resources. ## Authentication -_Libraries for implementing authentication schemes._ +_Libraries for implementing authentications schemes._ - OAuth - - [authlib](https://github.com/authlib/authlib) - JavaScript Object Signing and Encryption draft implementation. + - [authlib](https://github.com/lepture/authlib) - JavaScript Object Signing and Encryption draft implementation. - [django-allauth](https://github.com/pennersr/django-allauth) - Authentication app for Django that "just works." - [django-oauth-toolkit](https://github.com/django-oauth/django-oauth-toolkit) - OAuth 2 goodies for Django. - [oauthlib](https://github.com/oauthlib/oauthlib) - A generic and thorough implementation of the OAuth request-signing logic. @@ -412,139 +413,113 @@ _Libraries for connecting and operating databases._ _Databases implemented in Python._ -- [chromadb](https://github.com/chroma-core/chroma) - An open-source embedding database for building AI applications with embeddings and semantic search. -- [duckdb](https://github.com/duckdb/duckdb) - An in-process SQL OLAP database management system; optimized for analytics and fast queries, similar to SQLite but for analytical workloads. +- [chromadb](https://github.com/chroma-core/chroma) - An open-source embedding database for building AI applications with embeddings. - [pickledb](https://github.com/patx/pickledb) - A simple and lightweight key-value store for Python. - [tinydb](https://github.com/msiemens/tinydb) - A tiny, document-oriented database. -- [ZODB](https://github.com/zopefoundation/ZODB) - A native object database for Python. A key-value and object graph database. +- [zodb](https://github.com/zopefoundation/ZODB) - A native object database for Python. A key-value and object graph database. ## Caching _Libraries for caching data._ -- [cachetools](https://github.com/tkem/cachetools) - Extensible memoizing collections and decorators. -- [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. +- [beaker](https://github.com/bbangert/beaker) - A WSGI middleware library for sessions and caching. - [dogpile.cache](https://github.com/sqlalchemy/dogpile.cache) - dogpile.cache is a next generation replacement for Beaker made by the same authors. -- [python-diskcache](https://github.com/grantjenks/python-diskcache) - SQLite and file backed cache backend with faster lookups than memcached and redis. +- [django-cache-machine](https://github.com/django-cache-machine/django-cache-machine) - Automatic caching and invalidation for Django models. +- [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. +- [pymemcache](https://github.com/pinterest/pymemcache) - A comprehensive, fast, pure-Python memcached client. +- [redis-py](https://github.com/redis/redis-py) - The Python client for Redis. +- [regional](https://github.com/olivierverdier/regional) - Regional is a Python caching library. ## Search _Libraries and software for indexing and performing search queries on data._ +- [elasticsearch-py](https://github.com/elastic/elasticsearch-py) - The official low-level Python client for Elasticsearch. +- [elasticsearch-dsl-py](https://github.com/elastic/elasticsearch-dsl-py) - The official high-level Python client for Elasticsearch. - [django-haystack](https://github.com/django-haystack/django-haystack) - Modular search for Django. -- [elasticsearch-py](https://github.com/elastic/elasticsearch-py) - The official low-level Python client for [Elasticsearch](https://www.elastic.co/products/elasticsearch). -- [pysolr](https://github.com/django-haystack/pysolr) - A lightweight Python wrapper for [Apache Solr](https://lucene.apache.org/solr/). +- [whoosh](https://github.com/whoosh-community/whoosh) - A fast, pure Python search engine library. ## Serialization _Libraries for serializing complex data types._ - [marshmallow](https://github.com/marshmallow-code/marshmallow) - A lightweight library for converting complex objects to and from simple Python datatypes. -- [msgpack](https://github.com/msgpack/msgpack-python) - MessagePack serializer implementation for Python. -- [orjson](https://github.com/ijl/orjson) - Fast, correct JSON library. +- [pysimdjson](https://github.com/TkTech/pysimdjson) - A Python bindings for simdjson. +- [python-rapidjson](https://github.com/python-rapidjson/python-rapidjson) - A Python wrapper around RapidJSON. +- [ultrajson](https://github.com/ultrajson/ultrajson) - Fast JSON decoder and encoder written in C with Python bindings. **Data & Science** ## Data Analysis -_Libraries for data analysis._ +_Libraries for data analyzing._ -- General - - [aws-sdk-pandas](https://github.com/aws/aws-sdk-pandas) - Pandas on AWS. - - [datasette](https://github.com/simonw/datasette) - An open source multi-tool for exploring and publishing data. - - [desbordante](https://github.com/desbordante/desbordante-core/) - An open source data profiler for complex pattern discovery. - - [ibis](https://github.com/ibis-project/ibis) - A portable Python dataframe library with a single API for 20+ backends. - - [modin](https://github.com/modin-project/modin) - A drop-in pandas replacement that scales workflows by changing a single line of code. - - [pandas](https://github.com/pandas-dev/pandas) - A library providing high-performance, easy-to-use data structures and data analysis tools. - - [pathway](https://github.com/pathwaycom/pathway) - Real-time data processing framework for Python with reactive dataflows. - - [polars](https://github.com/pola-rs/polars) - A fast DataFrame library implemented in Rust with a Python API. -- Financial Data - - [akshare](https://github.com/akfamily/akshare) - A financial data interface library, built for human beings! - - [edgartools](https://github.com/dgunning/edgartools) - Library for downloading structured data from SEC EDGAR filings and XBRL financial statements. - - [openbb](https://github.com/OpenBB-finance/OpenBB) - A financial data platform for analysts, quants and AI agents. - - [yfinance](https://github.com/ranaroussi/yfinance) - Easy Pythonic way to download market and financial data from Yahoo Finance. +- [pandas](https://github.com/pandas-dev/pandas) - Flexible and powerful data analysis / manipulation library for Python. +- [polars](https://github.com/pola-rs/polars) - A blazingly fast DataFrame library. +- [optimus](https://github.com/hi-primus/optimus) - Agile Data Science Workflows made easy with PySpark. ## Data Validation _Libraries for validating data. Used for forms in many cases._ - [cerberus](https://github.com/pyeve/cerberus) - A lightweight and extensible data validation library. -- [jsonschema](https://github.com/python-jsonschema/jsonschema) - An implementation of [JSON Schema](http://json-schema.org/) for Python. -- [pandera](https://github.com/unionai-oss/pandera) - A data validation library for dataframes, with support for pandas, polars, and Spark. +- [colander](https://github.com/Pylons/colander) - Validating and deserializing data obtained via XML, JSON, an HTML Form post. +- [jsonschema](https://github.com/python-jsonschema/jsonschema) - An implementation of JSON Schema for Python. +- [schema](https://github.com/keleshev/schema) - A library for validating Python data structures. +- [schematics](https://github.com/schematics/schematics) - Data Structure Validation. +- [voluptuous](https://github.com/alecthomas/voluptuous) - A Python data validation library. - [pydantic](https://github.com/pydantic/pydantic) - Data validation using Python type hints. -- [voluptuous](https://github.com/alecthomas/voluptuous) - A Python data validation library primarily intended for validating data from untrusted sources. ## Data Visualization -_Libraries for visualizing data. Also see [awesome-javascript](https://github.com/sorrycc/awesome-javascript#data-visualization)._ +_Libraries for visualizing data._ -- Plotting - - [altair](https://github.com/vega/altair) - Declarative statistical visualization library for Python. - - [bokeh](https://github.com/bokeh/bokeh) - Interactive Web Plotting for Python. - - [bqplot](https://github.com/bqplot/bqplot) - Interactive Plotting Library for the Jupyter Notebook. - - [matplotlib](https://github.com/matplotlib/matplotlib) - A Python 2D plotting library. - - [plotly](https://github.com/plotly/plotly.py) - Interactive graphing library for Python. - - [plotnine](https://github.com/has2k1/plotnine) - A grammar of graphics for Python based on ggplot2. - - [pygal](https://github.com/Kozea/pygal) - A Python SVG Charts Creator. - - [pyqtgraph](https://github.com/pyqtgraph/pyqtgraph) - Interactive and realtime 2D/3D/Image plotting and science/engineering widgets. - - [seaborn](https://github.com/mwaskom/seaborn) - Statistical data visualization using Matplotlib. - - [ultraplot](https://github.com/ultraplot/UltraPlot) - Matplotlib wrapper for publication-ready scientific figures with minimal code. Includes advanced subplot management, panel layouts, and batteries-included geoscience plotting. - - [vispy](https://github.com/vispy/vispy) - High-performance scientific visualization based on OpenGL. -- Specialized - - [cartopy](https://github.com/SciTools/cartopy) - A cartographic python library with matplotlib support. - - [pygraphviz](https://github.com/pygraphviz/pygraphviz/) - Python interface to [Graphviz](http://www.graphviz.org/). -- Dashboards and Apps - - [gradio](https://github.com/gradio-app/gradio) - Build and share machine learning apps, all in Python. - - [streamlit](https://github.com/streamlit/streamlit) - A framework which lets you build dashboards, generate reports, or create chat apps in minutes. +- [altair](https://github.com/altair-viz/altair) - Declarative statistical visualization library for Python. +- [bokeh](https://github.com/bokeh/bokeh) - Interactive Web Plotting for Python. +- [bqplot](https://github.com/bloomberg/bqplot) - Interactive Plotting Library for the Jupyter Notebook. +- [cartopy](https://github.com/SciTools/cartopy) - A cartographic python library with matplotlib support. +- [diagrams](https://github.com/mingrammer/diagrams) - Diagram as Code. +- [matplotlib](https://github.com/matplotlib/matplotlib) - A Python 2D plotting library. +- [plotly](https://github.com/plotly/plotly.py) - Collaborative, interactive, publication-quality graphs. +- [pygal](https://github.com/Kozea/pygal) - A Python SVG Charts Creator. +- [pygraphviz](https://github.com/pygraphviz/pygraphviz/) - Python interface to Graphviz. +- [pyqtgraph](https://github.com/pyqtgraph/pyqtgraph) - Interactive and realtime 2D/3D/Image plotting and science/engineering widgets. +- [seaborn](https://github.com/mwaskom/seaborn) - Statistical data visualization using Matplotlib. +- [vispy](https://github.com/vispy/vispy) - High-performance scientific visualization based on OpenGL. ## Geolocation _Libraries for geocoding addresses and working with latitudes and longitudes._ -- [django-countries](https://github.com/SmileyChris/django-countries) - A Django app that provides a country field for models and forms. -- [geodjango](https://github.com/django/django) - A world-class geographic web framework that is part of [Django](https://docs.djangoproject.com/en/dev/ref/contrib/gis/). -- [geojson](https://github.com/jazzband/geojson) - Python bindings and utilities for GeoJSON. -- [geopandas](https://github.com/geopandas/geopandas) - Python tools for geographic data (GeoSeries/GeoDataFrame) built on pandas. +- [django-countries](https://github.com/SmileyChris/django-countries) - A Django app that provides country choices for use with forms, flag icons static files, and a country field for models. +- [geodjango](https://docs.djangoproject.com/en/dev/ref/contrib/gis/) - A world-class geographic web framework. - [geopy](https://github.com/geopy/geopy) - Python Geocoding Toolbox. +- [gpxpy](https://github.com/tkrajina/gpxpy) - A library for parsing and creating GPX files. +- [shapely](https://github.com/shapely/shapely) - Manipulation and analysis of geometric objects. ## Science -_Libraries for scientific computing. Also see [Python-for-Scientists](https://github.com/TomNicholas/Python-for-Scientists)._ +_Libraries for scientific computing._ -- Core - - [numba](https://github.com/numba/numba) - Python JIT compiler to LLVM aimed at scientific Python. - - [numpy](https://github.com/numpy/numpy) - A fundamental package for scientific computing with Python. - - [scipy](https://github.com/scipy/scipy) - A Python-based ecosystem of open-source software for mathematics, science, and engineering. - - [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. - - [sympy](https://github.com/sympy/sympy) - A Python library for symbolic mathematics. -- Biology and Chemistry - - [biopython](https://github.com/biopython/biopython) - Biopython is a set of freely available tools for biological computation. - - [cclib](https://github.com/cclib/cclib) - A library for parsing and interpreting the results of computational chemistry packages. - - [openbabel](https://github.com/openbabel/openbabel) - A chemical toolbox designed to speak the many languages of chemical data. - - [rdkit](https://github.com/rdkit/rdkit) - Cheminformatics and Machine Learning Software. -- Physics and Engineering - - [astropy](https://github.com/astropy/astropy) - A community Python library for Astronomy. - - [obspy](https://github.com/obspy/obspy) - A Python toolbox for seismology. - - [pydy](https://github.com/pydy/pydy) - Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion. - - [PythonRobotics](https://github.com/AtsushiSakai/PythonRobotics) - This is a compilation of various robotics algorithms with visualizations. -- Simulation and Modeling - - [pathsim](https://github.com/pathsim/pathsim) - A block-based system modeling and simulation framework with a browser-based visual editor. - - [pymc](https://github.com/pymc-devs/pymc) - Probabilistic programming and Bayesian modeling in Python. - - [simpy](https://gitlab.com/team-simpy/simpy) - A process-based discrete-event simulation framework. -- Other - - [colour](https://github.com/colour-science/colour) - Implementing a comprehensive number of colour theory transformations and algorithms. - - [manim](https://github.com/ManimCommunity/manim) - An animation engine for explanatory math videos. - - [networkx](https://github.com/networkx/networkx) - A high-productivity software for complex networks. - - [shapely](https://github.com/shapely/shapely) - Manipulation and analysis of geometric objects in the Cartesian plane. +- [astropy](https://github.com/astropy/astropy) - A community Python library for Astronomy. +- [biopython](https://github.com/biopython/biopython) - Python tools for computational molecular biology. +- [cclib](https://github.com/cclib/cclib) - Parsers and algorithms for computational chemistry logfiles. +- [nibabel](https://github.com/nipy/nibabel) - Provides read / write access to some common medical and neuroimaging file formats. +- [nilearn](https://github.com/nilearn/nilearn) - Machine learning for Neuro-Imaging in Python. +- [open-babel](https://github.com/openbabel/openbabel) - A chemical toolbox designed to speak the many languages of chemical data. +- [pymatgen](https://github.com/materialsproject/pymatgen) - A robust, open-source Python library for materials analysis. +- [rdkit](https://github.com/rdkit/rdkit) - Cheminformatics and Machine-Learning Software. +- [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. +- [sympy](https://github.com/sympy/sympy) - A Python library for symbolic mathematics. ## Quantum Computing -_Libraries for quantum computing._ +_Libraries for Quantum Computing._ -- [Cirq](https://github.com/quantumlib/Cirq) — A Google-developed framework focused on hardware-aware quantum circuit design for NISQ devices. -- [pennylane](https://github.com/PennyLaneAI/pennylane) — A hybrid quantum-classical machine learning library with automatic differentiation support. -- [qiskit](https://github.com/Qiskit/qiskit) — An IBM-backed quantum SDK for building, simulating, and running circuits on real quantum hardware. -- [qutip](https://github.com/qutip/qutip) - Quantum Toolbox in Python. +- [cirq](https://github.com/quantumlib/Cirq) - A python framework for creating, editing, and invoking Noisy Intermediate Scale Quantum (NISQ) circuits. +- [pennylane](https://github.com/PennyLaneAI/pennylane) - A cross-platform Python library for differentiable programming of quantum computers. +- [pyquil](https://github.com/rigetti/pyquil) - A Python library for quantum programming using Quil. +- [qiskit](https://github.com/Qiskit/qiskit-terra) - An open-source framework for working with noisy quantum computers at the level of pulses, circuits, and algorithms. **Developer Tools** @@ -553,103 +528,99 @@ _Libraries for quantum computing._ _Python implementation of data structures, algorithms and design patterns. Also see [awesome-algorithms](https://github.com/tayllan/awesome-algorithms)._ - Algorithms - - [algorithms](https://github.com/keon/algorithms) - Minimal examples of data structures and algorithms. + - [algorithms](https://github.com/keon/algorithms) - Minimal examples of data structures and algorithms in Python. + - [python-ds](https://github.com/prabhupant/python-ds) - A collection of data structure and algorithms for coding interviews. - [sortedcontainers](https://github.com/grantjenks/python-sortedcontainers) - Fast and pure-Python implementation of sorted collections. - [thealgorithms](https://github.com/TheAlgorithms/Python) - All Algorithms implemented in Python. - Design Patterns + - [pypattyrn](https://github.com/tylerlaberge/PyPattyrn) - A simple yet effective library for implementing common design patterns. - [python-patterns](https://github.com/faif/python-patterns) - A collection of design patterns in Python. - - [transitions](https://github.com/pytransitions/transitions) - A lightweight, object-oriented finite state machine implementation. + - [transitions](https://github.com/pytransitions/transitions) - A lightweight, object-oriented finite state machine implementation in Python. ## Interactive Interpreter _Interactive Python interpreters (REPL)._ -- [jupyter](https://github.com/jupyter/notebook) - A rich toolkit to help you make the most out of using Python interactively. +- [bpython](https://github.com/bpython/bpython) - A fancy interface to the Python interpreter. +- [ipython](https://github.com/ipython/ipython) - A rich toolkit to help you make the most out of using Python interactively. - [awesome-jupyter](https://github.com/markusschanta/awesome-jupyter) -- [marimo](https://github.com/marimo-team/marimo) - Transform data and train models, feels like a next-gen notebook, stored as Git-friendly Python. -- [ptpython](https://github.com/prompt-toolkit/ptpython) - Advanced Python REPL built on top of the [python-prompt-toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit). +- [ptpython](https://github.com/prompt-toolkit/python-prompt-toolkit) - Advanced Python REPL built on top of the python-prompt-toolkit. ## Code Analysis -_Tools of static analysis, linters and code quality checkers. Also see [awesome-static-analysis](https://github.com/analysis-tools-dev/static-analysis)._ +_Tools of static analysis, linters and code quality checkers._ -- Code Analysis - - [code2flow](https://github.com/scottrogowski/code2flow) - Turn your Python and JavaScript code into DOT flowcharts. - - [prospector](https://github.com/prospector-dev/prospector) - A tool to analyze Python code. - - [vulture](https://github.com/jendrikseipp/vulture) - A tool for finding and analyzing dead Python code. -- Code Linters - - [bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code. - - [flake8](https://github.com/PyCQA/flake8) - A wrapper around `pycodestyle`, `pyflakes` and McCabe. - - [awesome-flake8-extensions](https://github.com/DmytroLitvinov/awesome-flake8-extensions) - - [pylint](https://github.com/pylint-dev/pylint) - A fully customizable source code analyzer. - - [ruff](https://github.com/astral-sh/ruff) - An extremely fast Python linter and code formatter. -- Code Formatters - - [black](https://github.com/psf/black) - The uncompromising Python code formatter. - - [isort](https://github.com/PyCQA/isort) - A Python utility / library to sort imports. - - [ruff](https://github.com/astral-sh/ruff) - An extremely fast Python linter and code formatter. -- Refactoring - - [rope](https://github.com/python-rope/rope) - Rope is a python refactoring library. -- Type Checkers - [awesome-python-typing](https://github.com/typeddjango/awesome-python-typing) - - [mypy](https://github.com/python/mypy) - Check variable types during compile time. - - [pyre-check](https://github.com/facebook/pyre-check) - Performant type checking. - - [ty](https://github.com/astral-sh/ty) - An extremely fast Python type checker and language server. - - [typeshed](https://github.com/python/typeshed) - Collection of library stubs for Python, with static types. -- Type Annotations Generators - - [monkeytype](https://github.com/Instagram/MonkeyType) - A system for Python that generates static type annotations by collecting runtime types. - - [pytype](https://github.com/google/pytype) - Pytype checks and infers types for Python code - without requiring type annotations. +- [flake8](https://github.com/PyCQA/flake8) - A wrapper around pyflakes, pycodestyle and McCabe. + - [awesome-flake8-extensions](https://github.com/DmytroLitvinov/awesome-flake8-extensions) +- [pylint](https://github.com/pylint-dev/pylint) - A fully customizable source code analyzer. +- [pyflakes](https://github.com/PyCQA/pyflakes) - A simple program which checks Python source files for errors. +- [mypy](https://github.com/python/mypy) - Check variable types during compile time. +- [pycodestyle](https://github.com/PyCQA/pycodestyle) - (formerly `pep8`) - A tool to check your Python code against some of the style conventions in PEP 8. +- [pydocstyle](https://github.com/PyCQA/pydocstyle) - A static analysis tool for checking compliance with Python docstring conventions. +- [radon](https://github.com/rubik/radon) - A tool that computes various metrics from the source code. +- [vulture](https://github.com/jendrikseipp/vulture) - A tool for finding and analysing dead Python code. ## Testing _Libraries for testing codebases and generating test data._ -- Frameworks +- Testing Frameworks - [hypothesis](https://github.com/HypothesisWorks/hypothesis) - Hypothesis is an advanced Quickcheck style property based testing library. + - [nose2](https://github.com/nose-devs/nose2) - The successor to `nose`, based on `unittest2`. - [pytest](https://github.com/pytest-dev/pytest) - A mature full-featured Python testing tool. - - [robotframework](https://github.com/robotframework/robotframework) - A generic test automation framework. - - [scanapi](https://github.com/scanapi/scanapi) - Automated Testing and Documentation for your REST API. + - [awesome-pytest](https://github.com/augustogoulart/awesome-pytest) + - [robot](https://github.com/robotframework/robotframework) - A generic test automation framework. - [unittest](https://docs.python.org/3/library/unittest.html) - (Python standard library) Unit testing framework. - Test Runners - - [nox](https://github.com/wntrblm/nox) - Flexible test automation for Python. - - [tox](https://github.com/tox-dev/tox) - Auto builds and tests distributions in multiple Python versions + - [green](https://github.com/CleanCut/green) - A clean, colorful test runner. + - [nose2](https://github.com/nose-devs/nose2) - The successor to `nose`, based on `unittest2`. + - [tox](https://github.com/tox-dev/tox) - Auto builds and tests distributions in multiple Python versions. - GUI / Web Testing - [locust](https://github.com/locustio/locust) - Scalable user load testing tool written in Python. - - [playwright-python](https://github.com/microsoft/playwright-python) - Python version of the Playwright testing and automation library. + - [playwright](https://github.com/microsoft/playwright-python) - Python version of the Playwright automated testing and scraping library. - [pyautogui](https://github.com/asweigart/pyautogui) - PyAutoGUI is a cross-platform GUI automation Python module for human beings. - - [schemathesis](https://github.com/schemathesis/schemathesis) - A tool for automatic property-based testing of web applications built with Open API / Swagger specifications. - - [selenium](https://github.com/SeleniumHQ/selenium) - Python bindings for [Selenium](https://selenium.dev/) [WebDriver](https://selenium.dev/documentation/webdriver/). + - [selenium](https://github.com/SeleniumHQ/selenium/) - Python bindings for Selenium WebDriver. + - [splinter](https://github.com/cobrateam/splinter) - Open source tool for testing web applications. - Mock + - [doublex](https://pypi.org/project/doublex/) - Powerful test doubles framework for Python. - [freezegun](https://github.com/spulec/freezegun) - Travel through time by mocking the datetime module. - - [mock](https://docs.python.org/3/library/unittest.mock.html) - (Python standard library) A mocking and patching library. - - [mocket](https://github.com/mindflayer/python-mocket) - A socket mock framework with gevent/asyncio/SSL support. + - [httmock](https://github.com/patrys/httmock) - A mocking library for requests for Python 2.6+ and 3.2+. + - [httpretty](https://github.com/gabrielfalcao/HTTPretty) - HTTP request mock tool for Python. + - [moto](https://github.com/spulec/moto) - A library that allows you to easily mock out tests based on AWS Infrastructure. - [responses](https://github.com/getsentry/responses) - A utility library for mocking out the requests Python library. - [vcrpy](https://github.com/kevin1024/vcrpy) - Record and replay HTTP interactions on your tests. - Object Factories - [factory_boy](https://github.com/FactoryBoy/factory_boy) - A test fixtures replacement for Python. - - [polyfactory](https://github.com/litestar-org/polyfactory) - mock data generation library with support to classes (continuation of `pydantic-factories`) + - [mixer](https://github.com/klen/mixer) - Another fixtures replacement. Supported Django, Flask, SQLAlchemy, Peewee and etc. + - [model_bakery](https://github.com/model-bakers/model_bakery) - Object factory for Django (rename of model_mommy). - Code Coverage - - [coverage](https://github.com/coveragepy/coveragepy) - Code coverage measurement. + - [coverage](https://github.com/nedbat/coveragepy) - Code coverage measurement for Python. - Fake Data - [faker](https://github.com/joke2k/faker) - A Python package that generates fake data. - - [mimesis](https://github.com/lk-geimfari/mimesis) - is a Python library that help you generate fake data. + - [mimesis](https://github.com/lk-geimfari/mimesis) - A high performance fake data generator. + - [radar](https://pypi.org/project/radar/) - Generate random datetime / time. ## Debugging Tools _Libraries for debugging code._ - pdb-like Debugger - - [ipdb](https://github.com/gotcha/ipdb) - IPython-enabled [pdb](https://docs.python.org/3/library/pdb.html). + - [ipdb](https://github.com/gotcha/ipdb) - IPython-enabled pdb. + - [pdb++](https://github.com/pdbpp/pdbpp) - A better pdb, drop in replacement for pdb. - [pudb](https://github.com/inducer/pudb) - A full-screen, console-based Python debugger. - Tracing - - [manhole](https://github.com/ionelmc/python-manhole) - Debugging UNIX socket connections and present the stacktraces for all threads and an interactive prompt. + - [manhole](https://github.com/ionelmc/python-manhole) - Debugging UNIX socket connections and present all threads and waitress stacks and an interactive prompt. + - [pyringe](https://github.com/google/pyringe) - Debugger capable of attaching to and injecting code into Python processes. - [python-hunter](https://github.com/ionelmc/python-hunter) - A flexible code tracing toolkit. - Profiler - - [py-spy](https://github.com/benfred/py-spy) - A sampling profiler for Python programs. Written in Rust. - - [scalene](https://github.com/plasma-umass/scalene) - A high-performance, high-precision CPU, GPU, and memory profiler for Python. + - [line_profiler](https://github.com/pyutils/line_profiler) - Line-by-line profiling. + - [memory_profiler](https://github.com/pythonprofilers/memory_profiler) - Monitor Memory usage of Python code. + - [py-spy](https://github.com/benfred/py-spy) - A sampling profiler for Python programs. - Others - - [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) - Display various debug information for Django. + - [django-debug-toolbar](https://github.com/jazzband/django-debug-toolbar) - Display various debug information for Django. - [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to flask. - [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call. - - [memory_graph](https://github.com/bterwijn/memory_graph) - Visualize Python data at runtime to debug references, mutability, and aliasing. + - [pyelftools](https://github.com/eliben/pyelftools) - Parsing and analyzing ELF files and DWARF debugging information. ## Build Tools @@ -683,22 +654,15 @@ _Software and libraries for DevOps._ - [boto3](https://github.com/boto/boto3) - Python interface to Amazon Web Services. - Configuration Management - [ansible](https://github.com/ansible/ansible) - A radically simple IT automation platform. - - [cloudinit](https://github.com/canonical/cloud-init) - A multi-distribution package that handles early initialization of a cloud instance. - - [openstack](https://github.com/openstack/openstack) - Open source software for building private and public clouds. - - [pyinfra](https://github.com/pyinfra-dev/pyinfra) - A versatile CLI tools and python libraries to automate infrastructure. - - [saltstack](https://github.com/saltstack/salt) - Infrastructure automation and management system. -- Deployment - - [chalice](https://github.com/aws/chalice) - A Python serverless microframework for AWS. +- SSH-style Deployment - [fabric](https://github.com/fabric/fabric) - A simple, Pythonic tool for remote execution and deployment. -- Monitoring and Processes + - [cuisine](https://github.com/sebastien/cuisine) - Chef-like functionality for Fabric. +- Process Management + - [supervisor](https://github.com/Supervisor/supervisor) - A Process Control System. +- Monitoring - [psutil](https://github.com/giampaolo/psutil) - A cross-platform process and system utilities module. - - [sentry-python](https://github.com/getsentry/sentry-python) - Sentry SDK for Python. - - [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python. - - [supervisor](https://github.com/Supervisor/supervisor) - Supervisor process control system for UNIX. -- Other - - [borg](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption. - - [chaostoolkit](https://github.com/chaostoolkit/chaostoolkit) - A Chaos Engineering toolkit & Orchestration for Developers. - - [pre-commit](https://github.com/pre-commit/pre-commit) - A framework for managing and maintaining multi-language pre-commit hooks. +- Backup + - [borgbackup](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption. ## Distributed Computing @@ -707,10 +671,13 @@ _Frameworks and libraries for Distributed Computing._ - Batch Processing - [dask](https://github.com/dask/dask) - A flexible parallel computing library for analytic computing. - [luigi](https://github.com/spotify/luigi) - A module that helps you build complex pipelines of batch jobs. - - [mpi4py](https://github.com/mpi4py/mpi4py) - Python bindings for MPI. - - [pyspark](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/) Python API. - - [joblib](https://github.com/joblib/joblib) - A set of tools to provide lightweight pipelining in Python. - - [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the machine learning ecosystem. + - [mrjob](https://github.com/Yelp/mrjob) - Run MapReduce jobs on Hadoop or Amazon Web Services. + - [prefect](https://github.com/PrefectHQ/prefect) - A modern workflow orchestration framework. + - [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the ML ecosystem. + - [spark](https://github.com/apache/spark) - Apache Spark Python API. +- Stream Processing + - [faust](https://github.com/robinhood/faust) - A stream processing library, porting the ideas from Kafka Streams to Python. + - [streamparse](https://github.com/Parsely/streamparse) - Run Python code against real-time streams of data via [Apache Storm](http://storm.apache.org/). ## Task Queues @@ -719,27 +686,26 @@ _Libraries for working with task queues._ - [celery](https://github.com/celery/celery) - An asynchronous task queue/job queue based on distributed message passing. - [dramatiq](https://github.com/Bogdanp/dramatiq) - A fast and reliable background task processing library for Python 3. - [huey](https://github.com/coleifer/huey) - Little multi-threaded task queue. +- [mrq](https://github.com/pricingassistant/mrq) - A distributed worker task queue in Python using Redis & gevent. - [rq](https://github.com/rq/rq) - Simple job queues for Python. ## Job Schedulers _Libraries for scheduling jobs._ -- [airflow](https://github.com/apache/airflow) - Airflow is a platform to programmatically author, schedule and monitor workflows. -- [apscheduler](https://github.com/agronholm/apscheduler) - A light but powerful in-process task scheduler that lets you schedule functions. -- [dagster](https://github.com/dagster-io/dagster) - An orchestration platform for the development, production, and observation of data assets. -- [prefect](https://github.com/PrefectHQ/prefect) - A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines. +- [apscheduler](https://github.com/agronholm/apscheduler) - Another Python Scheduler. +- [django-celery-beat](https://github.com/celery/django-celery-beat) - A periodic task module for Celery. - [schedule](https://github.com/dbader/schedule) - Python job scheduling for humans. -- [SpiffWorkflow](https://github.com/sartography/SpiffWorkflow) - A powerful workflow engine implemented in pure Python. ## Logging _Libraries for generating and working with logs._ -- [logfmter](https://github.com/josheppinette/python-logfmter) - A standard library compatible logfmt formatter. -- [logging](https://docs.python.org/3/library/logging.html) - (Python standard library) Logging facility for Python. +- [logbook](https://github.com/getlogbook/logbook) - Logging replacement for Python. - [loguru](https://github.com/Delgan/loguru) - Library which aims to bring enjoyable logging in Python. -- [structlog](https://github.com/hynek/structlog) - Structured logging made easy. +- [pyzmq](https://github.com/zeromq/pyzmq) - A Python interface to ZeroMQ. +- [structlog](https://github.com/hynek/structlog) - Structured Logging for Python. +- [eliot](https://github.com/itamarst/eliot) - Logging for complex and distributed systems. ## Network Virtualization @@ -747,28 +713,20 @@ _Tools and libraries for Virtual Networking and SDN (Software Defined Networking - [mininet](https://github.com/mininet/mininet) - A popular network emulator and API written in Python. - [napalm](https://github.com/napalm-automation/napalm) - Cross-vendor API to manipulate network devices. -- [scapy](https://github.com/secdev/scapy) - A brilliant packet manipulation library. +- [pox](https://github.com/noxrepo/pox) - A Python-based SDN control applications, such as OpenFlow SDN controllers. **CLI & GUI** ## CLI Development -_Libraries for building command-line applications._ +_Libraries for building command-line application._ -- CLI Development - - [argparse](https://docs.python.org/3/library/argparse.html) - (Python standard library) Command-line option and argument parsing. - - [cement](https://github.com/datafolklabs/cement) - CLI Application Framework for Python. - - [click](https://github.com/pallets/click/) - A package for creating beautiful command line interfaces in a composable way. - - [python-fire](https://github.com/google/python-fire) - A library for creating command line interfaces from absolutely any Python object. - - [python-prompt-toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. - - [typer](https://github.com/fastapi/typer) - Modern CLI framework that uses Python type hints. Built on Click and Pydantic. -- Terminal Rendering - - [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of Progress Bar, with real-time throughput, eta and very cool animations. - - [asciimatics](https://github.com/peterbrittain/asciimatics) - A package to create full-screen text UIs (from interactive forms to ASCII animations). - - [colorama](https://github.com/tartley/colorama) - Cross-platform colored terminal text. - - [rich](https://github.com/Textualize/rich) - Python library for rich text and beautiful formatting in the terminal. Also provides a great `RichHandler` log handler. - - [textual](https://github.com/Textualize/textual) - A framework for building interactive user interfaces that run in the terminal and the browser. - - [tqdm](https://github.com/tqdm/tqdm) - Fast, extensible progress bar for loops and CLI. +- [cement](https://github.com/datafolklabs/cement) - CLI Application Framework for Python. +- [click](https://github.com/pallets/click) - A package for creating beautiful command line interfaces in a composable way. +- [cliff](https://github.com/openstack/cliff) - A framework for creating command-line programs with multi-level commands. +- [docopt](https://github.com/docopt/docopt) - Pythonic command line arguments parser. +- [python-fire](https://github.com/google/python-fire) - A library for creating command line interfaces from absolutely any Python object. +- [python-prompt-toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. ## CLI Tools @@ -777,6 +735,7 @@ _Useful CLI-based tools for productivity._ - Productivity Tools - [cookiecutter](https://github.com/cookiecutter/cookiecutter) - A command-line utility that creates projects from cookiecutters (project templates). - [copier](https://github.com/copier-org/copier) - A library and command-line utility for rendering projects templates. + - [structkit](https://github.com/httpdss/structkit) - A YAML-first project scaffolding tool with remote content fetching (GitHub, S3, GCS, HTTP) and MCP integration for AI-native workflows. - [doitlive](https://github.com/sloria/doitlive) - A tool for live presentations in the terminal. - [thefuck](https://github.com/nvbn/thefuck) - Correcting your previous console command. - [tmuxp](https://github.com/tmux-python/tmuxp) - A [tmux](https://github.com/tmux/tmux) session manager. @@ -793,27 +752,18 @@ _Useful CLI-based tools for productivity._ _Libraries for working with graphical user interface applications._ -- Desktop - - [customtkinter](https://github.com/tomschimansky/customtkinter) - A modern and customizable python UI-library based on Tkinter. - - [dearpygui](https://github.com/hoffstadt/DearPyGui) - A Simple GPU accelerated Python GUI framework - - [enaml](https://github.com/nucleic/enaml) - Creating beautiful user-interfaces with Declarative Syntax like QML. - - [kivy](https://github.com/kivy/kivy) - A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS. - - [pyglet](https://github.com/pyglet/pyglet) - A cross-platform windowing and multimedia library for Python. - - [pygobject](https://github.com/GNOME/pygobject) - Python Bindings for GLib/GObject/GIO/GTK+ (GTK+3). - - [PyQt](https://www.riverbankcomputing.com/static/Docs/PyQt6/) - Python bindings for the [Qt](https://www.qt.io/) cross-platform application and UI framework. - - [pyside](https://github.com/pyside/pyside-setup) - Qt for Python offers the official Python bindings for [Qt](https://www.qt.io/), this is same as PyQt but it's the official binding with different licensing. - - [tkinter](https://docs.python.org/3/library/tkinter.html) - (Python standard library) The standard Python interface to the Tcl/Tk GUI toolkit. - - [toga](https://github.com/beeware/toga) - A Python native, OS native GUI toolkit. - - [wxPython](https://github.com/wxWidgets/Phoenix) - A blending of the wxWidgets C++ class library with the Python. -- Web-based - - [flet](https://github.com/flet-dev/flet) - Cross-platform GUI framework for building modern apps in pure Python. - - [nicegui](https://github.com/zauberzeug/nicegui) - An easy-to-use, Python-based UI framework, which shows up in your web browser. - - [pywebview](https://github.com/r0x0r/pywebview/) - A lightweight cross-platform native wrapper around a webview component. -- Terminal - - [curses](https://docs.python.org/3/library/curses.html) - Built-in wrapper for [ncurses](http://www.gnu.org/software/ncurses/) used to create terminal GUI applications. - - [urwid](https://github.com/urwid/urwid) - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc. -- Wrappers - - [gooey](https://github.com/chriskiehl/Gooey) - Turn command line programs into a full GUI application with one line. +- [curses](https://docs.python.org/3/library/curses.html) - Built-in wrapper for [ncurses](http://www.gnu.org/software/ncurses/) used to create terminal GUI applications. +- [dearpygui](https://github.com/hoffstadt/DearPyGui) - A Simple to use (but powerful) GUI framework for Python. +- [eel](https://github.com/python-eel/Eel) - A little library for making simple Electron-like offline HTML/JS GUI apps. +- [flexx](https://github.com/flexxui/flexx) - Write desktop and web apps in pure Python. +- [kivy](https://github.com/kivy/kivy) - Open source software for creating NUI applications. +- [pyglet](https://github.com/pyglet/pyglet) - A cross-platform windowing and multimedia library for Python. +- [pygame](https://github.com/pygame/pygame) - A set of Python modules designed for writing games. +- [pyqt](https://doc.qt.io/qtforpython/) - Python bindings for the Qt cross-platform application and UI framework. +- [pyqtgraph](https://github.com/pyqtgraph/pyqtgraph) - Interactive and realtime 2D/3D/Image plotting and science/engineering widgets. +- [tkinter](https://wiki.python.org/moin/TkInter) - Tkinter is Python's de-facto standard GUI package. +- [toga](https://github.com/beeware/toga) - A Python native, OS native GUI toolkit. +- [wxpython](https://wxpython.org/) - A blending of the wxWidgets C++ class library with the Python. **Text & Documents** @@ -822,20 +772,23 @@ _Libraries for working with graphical user interface applications._ _Libraries for parsing and manipulating plain texts._ - General - - [babel](https://github.com/python-babel/babel) - An internationalization library for Python. - [chardet](https://github.com/chardet/chardet) - Python 2/3 compatible character encoding detector. - [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas. - - [ftfy](https://github.com/rspeer/python-ftfy) - Makes Unicode text less broken and more consistent automagically. - - [pangu.py](https://github.com/vinta/pangu.py) - Paranoid text spacing. - - [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python. - - [pypinyin](https://github.com/mozillazg/python-pinyin) - Convert Chinese hanzi (漢字) to pinyin (拼音). + - [ftfy](https://github.com/LuminosoInsight/python-ftfy) - Makes Unicode text less broken and more consistent automagically. + - [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) - Fuzzy String Matching. + - [Levenshtein](https://github.com/maxbachmann/Levenshtein) - Fast computation of Levenshtein distance and string similarity. + - [shortuuid](https://github.com/skorokithakis/shortuuid) - A generator library for concise, unambiguous and URL-safe UUIDs. + - [umalqurra](https://github.com/tytkal/python-hijiri-ummalqura) - Date Format for Arabic letters. + - [unidecode](https://pypi.org/project/Unidecode/) - ASCII transliterations of Unicode text. +- Slugify + - [awesome-slugify](https://github.com/dimka665/awesome-slugify) - A Python slugify library that can preserve unicode. - [python-slugify](https://github.com/un33k/python-slugify) - A Python slugify library that translates unicode to ASCII. - - [textdistance](https://github.com/life4/textdistance) - Compute distance between sequences with 30+ algorithms. - - [unidecode](https://github.com/avian2/unidecode) - ASCII transliterations of Unicode text. + - [unicode-slugify](https://github.com/mozilla/unicode-slugify) - A slugifier that generates unicode slugs with Django as a dependency. - Unique identifiers - - [sqids](https://github.com/sqids/sqids-python) - A library for generating short unique IDs from numbers. + - [hashids](https://github.com/davidaurelio/hashids-python) - Implementation of [hashids](http://hashids.org) in Python. - [shortuuid](https://github.com/skorokithakis/shortuuid) - A generator library for concise, unambiguous and URL-safe UUIDs. - Parser + - [ply](https://github.com/dabeaz/ply) - Implementation of lex and yacc parsing tools for Python. - [pygments](https://github.com/pygments/pygments) - A generic syntax highlighter. - [pyparsing](https://github.com/pyparsing/pyparsing) - A general purpose framework for generating parsers. - [python-nameparser](https://github.com/derek73/python-nameparser) - Parsing human names into their individual components. @@ -847,271 +800,282 @@ _Libraries for parsing and manipulating plain texts._ _Libraries for working with HTML and XML._ -- [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML. -- [justhtml](https://github.com/EmilStenstrom/justhtml/) - A pure Python HTML5 parser that just works. +- [beautifulsoup](https://beautiful-soup-4.readthedocs.io/en/latest/) - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML. +- [cssutils](https://pypi.org/project/cssutils/) - A CSS library for Python. +- [html5lib](https://github.com/html5lib/html5lib-python) - A standards-compliant library for parsing and serializing HTML documents and fragments. - [lxml](https://github.com/lxml/lxml) - A very fast, easy-to-use and versatile library for handling HTML and XML. - [markupsafe](https://github.com/pallets/markupsafe) - Implements a XML/HTML/XHTML Markup safe string for Python. - [pyquery](https://github.com/gawel/pyquery) - A jQuery-like library for parsing HTML. -- [tinycss2](https://github.com/Kozea/tinycss2) - A low-level CSS parser and generator written in Python. +- [untangle](https://github.com/stchris/untangle) - Converts XML documents to Python objects for easy access. +- [xmldataset](https://xmldataset.readthedocs.io/en/latest/) - Simple XML Parsing. - [xmltodict](https://github.com/martinblech/xmltodict) - Working with XML feel like you are working with JSON. -## File Format Processing +## PDF -_Libraries for parsing and manipulating specific text formats._ +_Libraries and software for working with PDF files._ -- General - - [docling](https://github.com/docling-project/docling) - Library for converting documents into structured data. - - [kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) - High-performance document extraction library with a Rust core, supporting 62+ formats including PDF, Office, images with OCR, HTML, email, and archives. - - [pyelftools](https://github.com/eliben/pyelftools) - Parsing and analyzing ELF files and DWARF debugging information. - - [tablib](https://github.com/jazzband/tablib) - A module for Tabular Datasets in XLS, CSV, JSON, YAML. -- MS Office - - [docxtpl](https://github.com/elapouya/python-docx-template) - Editing a docx document by jinja2 template - - [openpyxl](https://openpyxl.readthedocs.io/en/stable/) - A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files. - - [pyexcel](https://github.com/pyexcel/pyexcel) - Providing one API for reading, manipulating and writing csv, ods, xls, xlsx and xlsm files. - - [python-docx](https://github.com/python-openxml/python-docx) - Reads, queries and modifies Microsoft Word 2007/2008 docx files. - - [python-pptx](https://github.com/scanny/python-pptx) - Python library for creating and updating PowerPoint (.pptx) files. - - [xlsxwriter](https://github.com/jmcnamara/XlsxWriter) - A Python module for creating Excel .xlsx files. - - [xlwings](https://github.com/xlwings/xlwings) - A BSD-licensed library that makes it easy to call Python from Excel and vice versa. -- PDF - - [pdf_oxide](https://github.com/yfedoseev/pdf_oxide) - A fast PDF library for text extraction, image extraction, and markdown conversion, powered by Rust. - - [pdfminer.six](https://github.com/pdfminer/pdfminer.six) - Pdfminer.six is a community maintained fork of the original PDFMiner. - - [pikepdf](https://github.com/pikepdf/pikepdf) - A powerful library for reading and editing PDF files, based on qpdf. - - [pypdf](https://github.com/py-pdf/pypdf) - A library capable of splitting, merging, cropping, and transforming PDF pages. - - [reportlab](https://www.reportlab.com/opensource/) - Allowing Rapid creation of rich PDF documents. - - [weasyprint](https://github.com/Kozea/WeasyPrint) - A visual rendering engine for HTML and CSS that can export to PDF. -- Markdown - - [markdown-it-py](https://github.com/executablebooks/markdown-it-py) - Markdown parser with 100% CommonMark support, extensions, and syntax plugins. - - [markdown](https://github.com/waylan/Python-Markdown) - A Python implementation of John Gruber’s Markdown. - - [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown. - - [mistune](https://github.com/lepture/mistune) - Fastest and full featured pure Python parsers of Markdown. -- Data Formats - - [csvkit](https://github.com/wireservice/csvkit) - Utilities for converting to and working with CSV. - - [pyyaml](https://github.com/yaml/pyyaml) - YAML implementations for Python. - - [tomllib](https://docs.python.org/3/library/tomllib.html) - (Python standard library) Parse TOML files. +- [pdfminer.six](https://github.com/pdfminer/pdfminer.six) - Pdfminer.six is a community maintained fork of the original PDFMiner. +- [pypdf](https://github.com/py-pdf/pypdf) - A pure-python PDF library capable of splitting, merging and transforming PDF pages. +- [reportlab](https://www.reportlab.com/opensource/) - Allowing Rapid creation of rich PDF documents. +- [weasyprint](https://github.com/Kozea/WeasyPrint) - A visual rendering engine for HTML and CSS that can export to PDF. -## File Manipulation +## Office Documents -_Libraries for file manipulation._ +_Libraries for parsing and modifying office documents._ -- [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. -- [pathlib](https://docs.python.org/3/library/pathlib.html) - (Python standard library) A cross-platform, object-oriented path library. -- [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. -- [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. -- [watchfiles](https://github.com/samuelcolvin/watchfiles) - Simple, modern and fast file watching and code reload in python. +- [docxtpl](https://github.com/elapouya/python-docx-template) - Editing a docx document by jinja2 template. +- [openpyxl](https://openpyxl.readthedocs.io/en/stable/) - A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files. +- [pyexcel](https://github.com/pyexcel/pyexcel) - Providing one API for reading, manipulating and writing csv, ods, xls, xlsx and xlsm files. +- [python-docx](https://github.com/python-openxml/python-docx) - Reads, queries and modifies Microsoft Word 2007/2008 docx files. +- [python-pptx](https://github.com/scanny/python-pptx) - Python library for creating and updating PowerPoint (.pptx) files. +- [unoconv](https://github.com/unoconv/unoconv) - Convert between any document format supported by LibreOffice/OpenOffice. +- [xlrd](https://github.com/python-excel/xlrd) - Library for developers to extract data from Microsoft Excel spreadsheet files. +- [xlsxwriter](https://github.com/jmcnamara/XlsxWriter) - A Python module for creating Excel .xlsx files. +- [xlwings](https://github.com/xlwings/xlwings) - A BSD-licensed library that makes it easy to call Python from Excel and vice versa. +- [xlwt](https://github.com/python-excel/xlwt) - Library to write data and formatting information to older Excel files. -**Media** +## Markdown + +_Libraries for working with Markdown._ + +- [markdown](https://github.com/Python-Markdown/markdown) - Python implementation of John Gruber's Markdown. +- [markdown-it-py](https://github.com/executablebooks/markdown-it-py) - Markdown parser with 100% CommonMark support, extensions, syntax plugins and high performance. +- [marko](https://github.com/frostming/marko) - A markdown parser with high extensibility. +- [mistune](https://github.com/lepture/mistune) - Fastest and yet powerful Python Markdown parser. +- [pymarkdown](https://github.com/jackdewinter/pymarkdown) - Markdown linter/formatter. + +**Multimedia** + +## Audio + +_Libraries for manipulating audio and its metadata._ + +- [audioread](https://github.com/beetbox/audioread) - Cross-library audio decoding. +- [dejavu](https://github.com/worldveil/dejavu) - Audio fingerprinting and recognition. +- [librosa](https://github.com/librosa/librosa) - Python library for audio and music analysis. +- [mutagen](https://github.com/quodlibet/mutagen) - A Python module to handle audio metadata. +- [pyaudio](https://people.csail.mit.edu/hubert/pyaudio/) - PyAudio provides Python bindings for PortAudio. +- [pydub](https://github.com/jiaaro/pydub) - Manipulate audio with a simple and easy high level interface. +- [pyechonest](https://github.com/echonest/pyechonest) - Python client for the Echo Nest API. +- [tinytag](https://github.com/tinytag/tinytag) - Library for reading music meta data of audio files. ## Image Processing _Libraries for manipulating images._ -- [pillow](https://github.com/python-pillow/Pillow) - Pillow is the friendly [PIL](https://www.pythonware.com/products/pil/) fork. -- [pymatting](https://github.com/pymatting/pymatting) - A library for alpha matting. +- [imgaug](https://github.com/aleju/imgaug) - Image augmentation for machine learning experiments. +- [pillow](https://github.com/python-pillow/Pillow) - The friendly PIL fork. - [python-barcode](https://github.com/WhyNotHugo/python-barcode) - Create barcodes in Python with no extra dependencies. +- [pymatting](https://github.com/pymatting/pymatting) - A library for alpha matting. - [python-qrcode](https://github.com/lincolnloop/python-qrcode) - A pure Python QR Code generator. -- [pyvips](https://github.com/libvips/pyvips) - A fast image processing library with low memory needs. -- [scikit-image](https://github.com/scikit-image/scikit-image) - A Python library for (scientific) image processing. +- [quads](https://github.com/fogleman/Quads) - Computer art based on quadtrees. +- [scikit-image](https://github.com/scikit-image/scikit-image) - A collection of algorithms for image processing. - [thumbor](https://github.com/thumbor/thumbor) - A smart imaging service. It enables on-demand crop, re-sizing and flipping of images. -- [wand](https://github.com/emcconville/wand) - Python bindings for [MagickWand](https://www.imagemagick.org/script/magick-wand.php), C API for ImageMagick. +- [wand](https://github.com/emcconville/wand) - Python bindings for MagickWand, C API for ImageMagick. -## Audio & Video Processing +## Video -_Libraries for manipulating audio, video, and their metadata._ +_Libraries for manipulating video and GIFs._ -- Audio - - [gtts](https://github.com/pndurette/gTTS) - Python library and CLI tool for converting text to speech using Google Translate TTS. - - [librosa](https://github.com/librosa/librosa) - Python library for audio and music analysis. - - [matchering](https://github.com/sergree/matchering) - A library for automated reference audio mastering. - - [pydub](https://github.com/jiaaro/pydub) - Manipulate audio with a simple and easy high level interface. -- Video - - [moviepy](https://github.com/Zulko/moviepy) - A module for script-based movie editing with many formats, including animated GIFs. - - [vidgear](https://github.com/abhiTronix/vidgear) - Most Powerful multi-threaded Video Processing framework. -- Metadata - - [beets](https://github.com/beetbox/beets) - A music library manager and [MusicBrainz](https://musicbrainz.org/) tagger. - - [mutagen](https://github.com/quodlibet/mutagen) - A Python module to handle audio metadata. - - [tinytag](https://github.com/devsnd/tinytag) - A library for reading music meta data of MP3, OGG, FLAC and Wave files. +- [moviepy](https://github.com/Zulko/moviepy) - A module for script-based movie editing with many formats. +- [scikit-video](https://github.com/aizvorski/scikit-video) - Video processing routines for SciPy. +- [vidgear](https://github.com/abhiTronix/vidgear) - Most Powerful multi-threaded Video Processing framework. -## Game Development +**Cryptography & Security** -_Awesome game development libraries._ +## Cryptography -- [arcade](https://github.com/pythonarcade/arcade) - Arcade is a modern Python framework for crafting games with compelling graphics and sound. -- [panda3d](https://github.com/panda3d/panda3d) - 3D game engine developed by Disney. -- [py-sdl2](https://github.com/py-sdl/py-sdl2) - A ctypes based wrapper for the SDL2 library. -- [pygame](https://github.com/pygame/pygame) - Pygame is a set of Python modules designed for writing games. -- [pyopengl](https://github.com/mcfletch/pyopengl) - Python ctypes bindings for OpenGL and it's related APIs. -- [renpy](https://github.com/renpy/renpy) - A Visual Novel engine. +- [cryptography](https://github.com/pyca/cryptography) - A package designed to expose cryptographic recipes and primitives to Python developers. +- [paramiko](https://github.com/paramiko/paramiko) - The leading native Python SSHv2 protocol library. +- [pynacl](https://github.com/pyca/pynacl) - Python binding to the Networking and Cryptography (NaCl) library. -**Python Language** +## Security -## Implementations +_Tools and libraries for security_ -_Implementations of Python._ +- [bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code. +- [sqlmap](https://github.com/sqlmapproject/sqlmap) - Automatic SQL injection and database takeover tool. -- [cpython](https://github.com/python/cpython) - Default, most widely used implementation of the Python programming language written in C. -- [cython](https://github.com/cython/cython) - Optimizing Static Compiler for Python. -- [ironpython](https://github.com/IronLanguages/ironpython3) - Implementation of the Python programming language written in C#. -- [micropython](https://github.com/micropython/micropython) - A lean and efficient Python programming language implementation. -- [pyodide](https://github.com/pyodide/pyodide) - Python distribution for the browser and Node.js based on WebAssembly. -- [pypy](https://github.com/pypy/pypy) - A very fast and compliant implementation of the Python language. +**OS & System** -## Built-in Classes Enhancement +## OS -_Libraries for enhancing Python built-in classes._ +_Libraries that allow you to access specific functionality of the operating system._ -- [attrs](https://github.com/python-attrs/attrs) - Replacement for `__init__`, `__eq__`, `__repr__`, etc. boilerplate in class definitions. -- [bidict](https://github.com/jab/bidict) - Efficient, Pythonic bidirectional map data structures and related functionality. -- [box](https://github.com/cdgriffith/Box) - Python dictionaries with advanced dot notation access. +- [barman](https://github.com/EnterpriseDB/barman) - Backup and recovery manager for PostgreSQL. +- [opsy](https://github.com/rsmitty/opsy) - A Flask app which enables cross-environment monitoring data aggregation. +- [pywin32](https://github.com/mhammond/pywin32) - Python Extensions for Windows. +- [systemd](https://github.com/systemd/python-systemd) - Python bindings for systemd facilities, including journald and login. + +## Files + +_Libraries for file manipulation and MIME type detection._ + +- [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. +- [path](https://github.com/jaraco/path) - A module wrapper for os.path. +- [pathlib](https://docs.python.org/3/library/pathlib.html) - (Python standard library) An cross-platform, object-oriented path library. +- [pathspec](https://github.com/cpburnz/python-pathspec) - A utility library for gitignore style pattern matching of file paths. +- [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. +- [zxpy](https://github.com/tusharsadhwani/zxpy) - Run shell commands from within Python files. + +## Date and Time + +_Libraries for working with dates and times._ + +- [arrow](https://github.com/arrow-py/arrow) - A Python library that offers a sensible and human-friendly approach to creating, manipulating, formatting and converting dates, times and timestamps. +- [dateutil](https://github.com/dateutil/dateutil) - Extensions to the standard Python datetime module. +- [moment](https://github.com/zachwill/moment) - A Python library for dealing with dates/times. Inspired by [Moment.js](http://momentjs.com/). +- [pendulum](https://github.com/sdispater/pendulum) - Python datetimes made easy. +- [pytz](https://launchpad.net/pytz) - World timezone definitions, modern and historical. +- [when.py](https://github.com/dirn/When.py) - Providing user-friendly functions to help perform common date and time actions. + +## Internationalization + +_Libraries for working with i18n._ + +- [babel](https://github.com/python-babel/babel) - An integrated collection of utilities that assist in internationalizing and localizing Python applications. +- [pyicu](https://github.com/ovalhub/pyicu) - A wrapper of International Components for Unicode C++ library ([ICU](https://icu.unicode.org/)). + +## Compatibility + +_Libraries for migrating from Python 2 to 3._ + +- [modernize](https://github.com/PyCQA/modernize) - Modernizes Python code for eventual Python 3 migration. +- [six](https://github.com/benjaminp/six) - Python 2 and 3 compatibility utilities. + +## Event + +_Pythonic event system._ + +- [blinker](https://github.com/jek/blinker) - A fast Python in-process signal/event dispatching system. +- [kombu](https://github.com/celery/kombu) - A message queue library for Python. +- [pyinotify](https://github.com/seb-m/pyinotify) - A module for monitoring filesystems events on Linux systems based on inotify kernel subsystem. +- [pymitter](https://github.com/riga/pymitter) - A small Python port of the Node.js EventEmitter. +- [python-socketio](https://github.com/miguelgrinberg/python-socketio) - A Socket.IO server implemented in Python. +- [zope.event](https://github.com/zopefoundation/zope.event) - An event publishing system for Python. + +## Process + +_Libraries for starting and communicating with OS processes._ + +- [dagger](https://github.com/dagger/dagger) - An engine to run your pipelines in containers. +- [delegator.py](https://github.com/amitt001/delegator.py) - [Subprocesses](https://docs.python.org/3/library/subprocess.html) for Humans 2.0. +- [sarge](https://github.com/vinay.sajip/sarge) - Yet another wrapper for subprocess. +- [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python. + +## Concurrency and Parallelism + +_Libraries for concurrent and parallel execution. Also see [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio)._ + +- [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) - (Python standard library) A high-level interface for asynchronously executing callables. +- [eventlet](https://github.com/eventlet/eventlet/) - Asynchronous framework with WSGI support. +- [gevent](https://github.com/gevent/gevent) - A coroutine-based Python networking library that uses greenlet. +- [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) - (Python standard library) Process-based parallelism. +- [trio](https://github.com/python-trio/trio) - A friendly library for async concurrency and I/O. +- [twisted](https://github.com/twisted/twisted) - An event-driven networking engine. +- [uvloop](https://github.com/MagicStack/uvloop) - Ultra fast asyncio event loop. + +**Other** + +## Configuration Files + +_Libraries for storing and parsing configuration options._ + +- [configobj](https://github.com/DiffSteam/configobj) - Config file reader and writer for Python. +- [configparser](https://docs.python.org/3/library/configparser.html) - (Python standard library) INI file parser. +- [dynaconf](https://github.com/rochacbruno/dynaconf) - A dynamic data structures based Python settings. +- [hydra](https://github.com/facebookresearch/hydra) - Hydra is a framework for elegantly configuring complex applications. +- [profig](https://github.com/dhain/profig) - Config from multiple formats with value conversion. +- [python-decouple](https://github.com/HBNetwork/python-decouple) - Strict separation of config from code. + +## Design Patterns + +_Libraries for implementing design patterns._ + +- [pypattyrn](https://github.com/tylerlaberge/PyPattyrn) - A simple yet effective library for implementing common design patterns. +- [python-patterns](https://github.com/faif/python-patterns) - A collection of design patterns in Python. +- [transitions](https://github.com/pytransitions/transitions) - A lightweight, object-oriented finite state machine implementation in Python. ## Functional Programming _Functional Programming with Python._ - [coconut](https://github.com/evhub/coconut) - A variant of Python built for simple, elegant, Pythonic functional programming. -- [functools](https://docs.python.org/3/library/functools.html) - (Python standard library) Higher-order functions and operations on callable objects. +- [cytoolz](https://github.com/pytoolz/cytoolz/) - Cython implementation of Toolz: High performance functional utilities. +- [fn.py](https://github.com/kachayev/fn.py) - Functional programming in Python: implementation of missing features to enjoy FP. - [funcy](https://github.com/Suor/funcy) - A fancy and practical functional tools. -- [more-itertools](https://github.com/erikrose/more-itertools) - More routines for operating on iterables, beyond `itertools`. -- [returns](https://github.com/dry-python/returns) - A set of type-safe monads, transformers, and composition utilities. -- [toolz](https://github.com/pytoolz/toolz) - A collection of functional utilities for iterators, functions, and dictionaries. Also available as [cytoolz](https://github.com/pytoolz/cytoolz/) for Cython-accelerated performance. +- [more-itertools](https://github.com/more-itertools/more-itertools) - More routines for operating on iterables, beyond itertools. +- [returns](https://github.com/dry-python/returns) - A set of primitives to write declarative business logic. +- [toolz](https://github.com/pytoolz/toolz) - A collection of functional utilities for iterables, functions, and dictionaries. -## Asynchronous Programming +## Game Development -_Libraries for asynchronous, concurrent and parallel execution. Also see [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio)._ +_Awesome game development libraries._ -- [anyio](https://github.com/agronholm/anyio) - A high-level async concurrency and networking framework that works on top of asyncio or trio. -- [asyncio](https://docs.python.org/3/library/asyncio.html) - (Python standard library) Asynchronous I/O, event loop, coroutines and tasks. - - [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio) -- [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) - (Python standard library) A high-level interface for asynchronously executing callables. -- [gevent](https://github.com/gevent/gevent) - A coroutine-based Python networking library that uses [greenlet](https://github.com/python-greenlet/greenlet). -- [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) - (Python standard library) Process-based parallelism. -- [trio](https://github.com/python-trio/trio) - A friendly library for async concurrency and I/O. -- [twisted](https://github.com/twisted/twisted) - An event-driven networking engine. -- [uvloop](https://github.com/MagicStack/uvloop) - Ultra fast asyncio event loop. - -## Date and Time - -_Libraries for working with dates and times._ - -- [dateparser](https://github.com/scrapinghub/dateparser) - A Python parser for human-readable dates in dozens of languages. -- [dateutil](https://github.com/dateutil/dateutil) - Extensions to the standard Python [datetime](https://docs.python.org/3/library/datetime.html) module. -- [pendulum](https://github.com/python-pendulum/pendulum) - Python datetimes made easy. -- [zoneinfo](https://docs.python.org/3/library/zoneinfo.html) - (Python standard library) IANA time zone support. Brings the [tz database](https://en.wikipedia.org/wiki/Tz_database) into Python. - -**Python Toolchain** - -## Environment Management - -_Libraries for Python version and virtual environment management._ - -- [pyenv](https://github.com/pyenv/pyenv) - Simple Python version management. -- [pyenv-win](https://github.com/pyenv-win/pyenv-win) - Pyenv for Windows. -- [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. -- [virtualenv](https://github.com/pypa/virtualenv) - A tool to create isolated Python environments. - -## Package Management - -_Libraries for package and dependency management._ - -- [conda](https://github.com/conda/conda/) - Cross-platform, Python-agnostic binary package manager. -- [pip](https://github.com/pypa/pip) - The package installer for Python. -- [pipx](https://github.com/pypa/pipx) - Install and Run Python Applications in Isolated Environments. Like `npx` in Node.js. -- [poetry](https://github.com/python-poetry/poetry) - Python dependency management and packaging made easy. -- [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. - -## Package Repositories - -_Local PyPI repository server and proxies._ - -- [bandersnatch](https://github.com/pypa/bandersnatch/) - PyPI mirroring tool provided by Python Packaging Authority (PyPA). -- [devpi](https://github.com/devpi/devpi) - PyPI server and packaging/testing/release tool. -- [warehouse](https://github.com/pypa/warehouse) - Next generation Python Package Repository (PyPI). - -## Distribution - -_Libraries to create packaged executables for release distribution._ - -- [cx-Freeze](https://github.com/marcelotduarte/cx_Freeze) - It is a Python tool that converts Python scripts into standalone executables and installers for Windows, macOS, and Linux. -- [Nuitka](https://github.com/Nuitka/Nuitka) - Compiles Python programs into high-performance standalone executables (cross-platform, supports all Python versions). -- [pyarmor](https://github.com/dashingsoft/pyarmor) - A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts. -- [pyinstaller](https://github.com/pyinstaller/pyinstaller) - Converts Python programs into stand-alone executables (cross-platform). -- [shiv](https://github.com/linkedin/shiv) - A command line utility for building fully self-contained zipapps (PEP 441), but with all their dependencies included. - -## Configuration Files - -_Libraries for storing and parsing configuration options._ - -- [configparser](https://docs.python.org/3/library/configparser.html) - (Python standard library) INI file parser. -- [dynaconf](https://github.com/dynaconf/dynaconf) - Dynaconf is a configuration manager with plugins for Django, Flask and FastAPI. -- [hydra](https://github.com/facebookresearch/hydra) - Hydra is a framework for elegantly configuring complex applications. -- [python-decouple](https://github.com/HBNetwork/python-decouple) - Strict separation of settings from code. -- [python-dotenv](https://github.com/theskumar/python-dotenv) - Reads key-value pairs from a `.env` file and sets them as environment variables. - -**Security** - -## Cryptography - -- [cryptography](https://github.com/pyca/cryptography) - A package designed to expose cryptographic primitives and recipes to Python developers. -- [paramiko](https://github.com/paramiko/paramiko) - The leading native Python SSHv2 protocol library. -- [pynacl](https://github.com/pyca/pynacl) - Python binding to the Networking and Cryptography (NaCl) library. - -## Penetration Testing - -_Frameworks and tools for penetration testing._ - -- [mitmproxy](https://github.com/mitmproxy/mitmproxy) - An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers. -- [setoolkit](https://github.com/trustedsec/social-engineer-toolkit) - A toolkit for social engineering. -- [sherlock](https://github.com/sherlock-project/sherlock) - Hunt down social media accounts by username across social networks. -- [sqlmap](https://github.com/sqlmapproject/sqlmap) - Automatic SQL injection and database takeover tool. - -**Miscellaneous** +- [arcade](https://github.com/pythonarcade/arcade) - Easy to use Python library for creating 2D arcade games. +- [cocos2d](https://github.com/los-cocos/cocos) - cocos2d is a framework for building 2D games, demos, and other graphical/interactive applications. +- [pygame](https://github.com/pygame/pygame) - A set of Python modules designed for writing games. +- [pyglet](https://github.com/pyglet/pyglet) - A cross-platform windowing and multimedia library for Python. +- [pysdl2](https://github.com/py-sdl/py-sdl2) - A wrapper around the SDL2 library. ## Hardware _Libraries for programming with hardware._ -- [bleak](https://github.com/hbldh/bleak) - A cross platform Bluetooth Low Energy Client for Python using asyncio. +- [ino](https://github.com/amperka/ino) - Command line toolkit for working with Arduino. +- [keyboard](https://github.com/boppreh/keyboard) - Hook and simulate global keyboard events on Windows and Linux. +- [mouse](https://github.com/boppreh/mouse) - Hook and simulate global mouse events on Windows and Linux. - [pynput](https://github.com/moses-palmer/pynput) - A library to control and monitor input devices. +- [pyserial](https://github.com/pyserial/pyserial) - Python serial port access library. +- [raspberry-gpio-python](https://sourceforge.net/p/raspberry-gpio-python/wiki/Home/) - A module to control Raspberry Pi GPIO channels. +- [scapy](https://github.com/secdev/scapy) - A brilliant packet manipulation library. -## Microsoft Windows +## Package Management -_Python programming on Microsoft Windows._ +_Libraries for package and dependency management._ -- [pythonnet](https://github.com/pythonnet/pythonnet) - Python Integration with the .NET Common Language Runtime (CLR). -- [pywin32](https://github.com/mhammond/pywin32) - Python Extensions for Windows. -- [winpython](https://github.com/winpython/winpython) - Portable development environment for Windows 10/11. +- [pip](https://pip.pypa.io/en/stable/) - The package installer for Python. + - [PyPI](https://pypi.org/) - Python Package Index. + - [pip-tools](https://github.com/jazzband/pip-tools) - A set of tools to keep your pinned Python dependencies fresh. +- [poetry](https://github.com/python-poetry/poetry) - Python dependency management and packaging made easy. +- [uv](https://github.com/astral-sh/uv) - An extremely fast Python package installer and resolver, written in Rust. -## Miscellaneous +## Package Repositories -_Useful libraries or tools that don't fit in the categories above._ +_Local PyPI repository server and proxies._ -- [blinker](https://github.com/jek/blinker) - A fast Python in-process signal/event dispatching system. -- [boltons](https://github.com/mahmoud/boltons) - A set of pure-Python utilities. -- [itsdangerous](https://github.com/pallets/itsdangerous) - Various helpers to pass trusted data to untrusted environments. -- [tryton](https://github.com/tryton/tryton) - A general-purpose business framework. +- [bandersnatch](https://github.com/pypa/bandersnatch/) - PyPI mirror tool provided by PyPA. +- [devpi](https://github.com/devpi/devpi) - PyPI server and packaging/testing/release tool. +- [localshop](https://github.com/jazzband/localshop) - Local PyPI server (custom packages and auto-mirroring of pypi). -# Resources +## Robotics -Where to discover learning resources or new Python libraries. +_Libraries for robotics._ -## Newsletters +- [PythonRobotics](https://github.com/AtsushiSakai/PythonRobotics) - This is a compilation of various robotics algorithms with visualizations. +- [rospy](https://github.com/ros/ros_comm) - ROS Python client library. -- [Awesome Python Newsletter](http://python.libhunt.com/newsletter) -- [Pycoder's Weekly](https://pycoders.com/) -- [Python Tricks](https://realpython.com/python-tricks/) -- [Python Weekly](https://www.pythonweekly.com/) +## RPC Servers -## Podcasts +_RPC-compatible servers._ -- [Django Chat](https://djangochat.com/) -- [PyPodcats](https://pypodcats.live) -- [Python Bytes](https://pythonbytes.fm) -- [Talk Python To Me](https://talkpython.fm/) -- [The Real Python Podcast](https://realpython.com/podcasts/rpp/) +- [RPyC](https://github.com/tomerfiliba-org/rpyc) (Remote Python Call) - A transparent and symmetric RPC library for Python. +- [zeroRPC](https://github.com/0rpc/zerorpc-python) - zerorpc is a flexible RPC implementation based on ZeroMQ and MessagePack. -# Contributing +## Websocket -Your contributions are always welcome! Please take a look at the [contribution guidelines](https://github.com/vinta/awesome-python/blob/master/CONTRIBUTING.md) first. +_Libraries for working with WebSocket._ ---- +- [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and asyncio. +- [channels](https://github.com/django/channels) - Developer-friendly asynchronous support for Django. +- [websockets](https://github.com/aaugustin/websockets) - A library for building WebSocket servers and clients focusing on correctness and simplicity. -If you have any question about this opinionated list, do not hesitate to contact [@vinta](https://x.com/vinta) on X (Twitter). +## WSGI Servers + +_WSGI-compatible web servers._ + +- [bjoern](https://github.com/jonashaag/bjoern) - Asynchronous, very fast and written in C. +- [gunicorn](https://github.com/benoitc/gunicorn) - Pre-forked, porting to Gevent and others. +- [uWSGI](https://uwsgi-docs.readthedocs.io/en/latest/) - A project aims at developing a full stack for building hosting services. +- [waitress](https://github.com/Pylons/waitress) - Multi-threaded, powers Pyramid. +- [werkzeug](https://github.com/pallets/werkzeug) - A comprehensive WSGI web application library.