awesome-nlp

keon / awesome-nlp

一个精选自然语言处理（NLP）资源列表，涵盖论文、工具、库和数据集，帮助开发者系统学习和应用NLP技术。

数据科学 AI 基础设施 开发工具 自然语言处理 资源列表 机器学习 文本分析 NLP工具

为什么值得看

编辑点评

这是NLP领域最经典的综合资源清单之一，覆盖从经典方法到前沿研究的广泛内容，适合初学者入门和研究者检索。优点在于分类清晰、社区维护活跃；缺点是链接较多，需自行筛选质量。适合想系统了解NLP生态、寻找特定工具或论文的开发者。

Star 趋势

近 7 日
  • Star 总数19,003
  • 今日新增+2
  • 7 日增速-80%
  • Fork2,869

同类项目

同场景 · 数据科学 / AI 基础设施 / 开发工具
openclaw
openclaw
TypeScript▲ 47
n8n
n8n-io
TypeScript▲ 100
tensorflow
tensorflow
C++▲ 18
yt-dlp
yt-dlp
Python▲ 107

项目文档

来自 GitHub README · main 分支

awesome-nlp

Awesome

Sponsored by Atlas Cloud

AI API aggregation platform with an OpenAI-compatible LLM endpoint for NLP tasks such as translation, summarization, multilingual generation, and structured extraction.

A curated list of resources dedicated to Natural Language Processing

Please read the contribution guidelines before contributing. Please add your favourite NLP resource by raising a pull request

Scope

This list covers natural language processing — linguistic analysis, multilingual tooling, classical and neural methods, datasets, and evaluation. Large language models are included only where they advance or evaluate a core NLP task or capability (tokenization, multilinguality, MT, summarization, NER, QA, factuality, probing, distillation). General-purpose chatbots, agent frameworks, prompt-template repositories, code-generation tools, and RAG application starter kits live in other lists — see See Also.

Contents

Where to follow current NLP research:

Historical highlights

Prominent NLP Research Labs

Back to Top

Tutorials

Back to Top

Reading Content

General Machine Learning

Introductions and Guides to NLP

Blogs and Newsletters

Videos and Online Courses

Back to Top

Books

Libraries

Back to Top

  • Node.js and Javascript - Node.js Libaries for NLP | Back to Top
  • Twitter-text - A JavaScript implementation of Twitter's text processing library
  • Knwl.js - A Natural Language Processor in JS
  • Retext - Extensible system for analyzing and manipulating natural language
  • NLP Compromise - Natural Language processing in the browser
  • Natural - general natural language facilities for node
  • Poplar - A web-based annotation tool for natural language processing (NLP)
  • NLP.js - An NLP library for building bots

  • node-question-answering - Fast and production-ready question answering w/ DistilBERT in Node.js

  • Python - Python NLP Libraries | Back to Top

  • sentimental-onix Sentiment models for spacy using onnx
  • TextAttack - Adversarial attacks, adversarial training, and data augmentation in NLP
  • TextBlob - Providing a consistent API for diving into common natural language processing (NLP) tasks. Stands on the giant shoulders of Natural Language Toolkit (NLTK) and Pattern, and plays nicely with both :+1:
  • spaCy - Industrial strength NLP with Python and Cython :+1:
    • textacy - Higher level NLP built on spaCy
  • gensim - Python library to conduct unsupervised semantic modelling from plain text :+1:
  • scattertext - Python library to produce d3 visualizations of how language differs between corpora
  • GluonNLP (archived) - A deep learning toolkit for NLP, built on MXNet/Gluon.
  • AllenNLP (archived) - An NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks.
  • PyTorch-NLP - NLP research toolkit designed to support rapid prototyping with better data loaders, word vector loaders, neural network layer representations, common NLP metrics such as BLEU
  • Rosetta - Text processing tools and wrappers (e.g. Vowpal Wabbit)
  • PyNLPl - Python Natural Language Processing Library. General purpose NLP library for Python, handles some specific formats like ARPA language models, Moses phrasetables, GIZA++ alignments.
  • foliapy - Python library for working with FoLiA, an XML format for linguistic annotation.
  • PySS3 - Python package implementing the SS3 white-box text classifier; ships with interactive visualization tools that explain predictions.
  • jPTDP - A toolkit for joint part-of-speech (POS) tagging and dependency parsing. jPTDP provides pre-trained models for 40+ languages.
  • BigARTM - a fast library for topic modelling
  • Snips NLU - A production ready library for intent parsing
  • Chazutsu - A library for downloading&parsing standard NLP research datasets
  • Word Forms - Word forms can accurately generate all possible forms of an English word
  • Multilingual Latent Dirichlet Allocation (LDA) - A multilingual and extensible document clustering pipeline
  • Natural Language Toolkit (NLTK) - A library containing a wide variety of NLP functionality, supporting over 50 corpora.
  • NLP Architect - A library for exploring the state-of-the-art deep learning topologies and techniques for NLP and NLU
  • Flair - A very simple framework for state-of-the-art multilingual NLP built on PyTorch. Includes BERT, ELMo and Flair embeddings.
  • Kashgari - Simple, Keras-powered multilingual NLP framework, allows you to build your models in 5 minutes for named entity recognition (NER), part-of-speech tagging (PoS) and text classification tasks. Includes BERT and word2vec embedding.
  • FARM - Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.
  • Haystack - End-to-end Python framework for building natural language search interfaces to data. Leverages Transformers and the State-of-the-Art of NLP. Supports DPR, Elasticsearch, HuggingFace’s Modelhub, and much more!
  • Rita DSL - a DSL, loosely based on RUTA on Apache UIMA. Allows to define language patterns (rule-based NLP) which are then translated into spaCy, or if you prefer less features and lightweight - regex patterns.
  • Transformers - Natural Language Processing for TensorFlow 2.0 and PyTorch.
  • Tokenizers - Tokenizers optimized for Research and Production.
  • fairSeq Facebook AI Research implementations of SOTA seq2seq models in Pytorch.
  • corex_topic - Hierarchical Topic Modeling with Minimal Domain Knowledge
  • Sockeye - Neural Machine Translation (NMT) toolkit that powers Amazon Translate.
  • DL Translate - A deep learning-based translation library for 50 languages, built on transformers and Facebook's mBART Large.
  • Jury - Evaluation of NLP model outputs offering various automated metrics.
  • python-ucto - Unicode-aware regular-expression based tokenizer for various languages. Python binding to C++ library, supports FoLiA format.
  • Pearmut - Human annotation tool for multilingual NLP tasks, such as machine translation.
  • Stanza - Stanford NLP's Python toolkit for tokenization, POS, lemma, dependency parsing, and NER across 70+ languages.
  • Sentence-Transformers - sentence/document embeddings, semantic search, and re-ranking; current standard for retrieval-style NLP.
  • Argilla - open-source data annotation and feedback collection platform for LLM and NLP datasets.
  • HuggingFace Datasets - standardized loaders and processing for thousands of NLP datasets.
  • HuggingFace Evaluate - reference implementations for NLP metrics.
  • sacrebleu - reproducible BLEU/chrF/TER scoring for machine translation.
  • COMET - learned MT metrics, current de-facto standard.
  • LangTest - 60+ test types for NLP model robustness, bias, and fairness.

  • yasbd-lib - High-accuracy, rule-based sentence boundary detector (SBD). Drop-in pysbd adapter, streaming APIs, CLI, and a spaCy component across 39+ languages.

  • C++ - C++ Libraries | Back to Top

  • InsNet - A neural network library for building instance-dependent NLP models with padding-free dynamic batching.
  • MIT Information Extraction Toolkit - C, C++, and Python tools for named entity recognition and relation extraction
  • CRF++ - Open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data & other Natural Language Processing tasks.
  • CRFsuite - CRFsuite is an implementation of Conditional Random Fields (CRFs) for labeling sequential data.
  • BLLIP Parser - BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)
  • colibri-core - C++ library, command line tools, and Python binding for extracting and working with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.
  • ucto - Unicode-aware regular-expression based tokenizer for various languages. Tool and C++ library. Supports FoLiA format.
  • libfolia - C++ library for the FoLiA format
  • frog - Memory-based NLP suite developed for Dutch: PoS tagger, lemmatiser, dependency parser, NER, shallow parser, morphological analyzer.
  • MeTA - ModErn Text Analysis: a C++ data sciences toolkit for mining big text data.
  • Mecab (Japanese)
  • Moses
  • StarSpace - a library from Facebook for creating embeddings of word-level, paragraph-level, document-level and for text classification

  • QSMM - adaptive probabilistic top-down and bottom-up parsers

  • Java - Java NLP Libraries | Back to Top

  • Stanford NLP
  • OpenNLP
  • NLP4J
  • Word2vec in Java
  • ReVerb Web-Scale Open Information Extraction
  • OpenRegex An efficient and flexible token-based regular expression language and engine.
  • CogcompNLP - Core libraries developed in the U of Illinois' Cognitive Computation Group.
  • MALLET - MAchine Learning for LanguagE Toolkit - package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.

  • RDRPOSTagger - A robust POS tagging toolkit available (in both Java & Python) together with pre-trained models for 40+ languages.

  • Kotlin - Kotlin NLP Libraries | Back to Top

  • Lingua A language detection library for Kotlin and Java, suitable for long and short text alike

  • Kotidgy — an index-based text data generator written in Kotlin

  • Scala - Scala NLP Libraries | Back to Top

  • Saul - Library for developing NLP systems, including built in modules like SRL, POS, etc.
  • ATR4S - Toolkit with state-of-the-art automatic term recognition methods.
  • tm - Implementation of topic modeling based on regularized multilingual PLSA.
  • word2vec-scala - Scala interface to word2vec model; includes operations on vectors like word-distance and word-analogy.
  • Epic - Epic is a high performance statistical parser written in Scala, along with a framework for building complex structured prediction models.

  • Spark NLP - Spark NLP is a natural language processing library built on top of Apache Spark ML that provides simple, performant & accurate NLP annotations for machine learning pipelines that scale easily in a distributed environment.

  • R - R NLP Libraries | Back to Top

  • text2vec - Fast vectorization, topic modeling, distances and GloVe word embeddings in R.
  • wordVectors - An R package for creating and exploring word2vec and other word embedding models
  • RMallet - R package to interface with the Java machine learning tool MALLET
  • dfr-browser - Creates d3 visualizations for browsing topic models of text in a web browser.
  • dfrtopics - R package for exploring topic models of text.
  • sentiment_classifier - Sentiment Classification using Word Sense Disambiguation and WordNet Reader
  • jProcessing - Japanese Natural Langauge Processing Libraries, with Japanese sentiment classification
  • corporaexplorer - An R package for dynamic exploration of text collections
  • tidytext - Text mining using tidy tools
  • spacyr - R wrapper to spaCy NLP

  • CRAN Task View: Natural Language Processing

  • Clojure | Back to Top

  • Clojure-openNLP - Natural Language Processing in Clojure (opennlp)
  • Infections-clj - Rails-like inflection library for Clojure and ClojureScript

  • postagga - A library to parse natural language in Clojure and ClojureScript

  • Go | Back to Top

  • prose - Text processing library supporting tokenization, part-of-speech tagging, and named-entity extraction.
  • gojieba - Go implementation of the jieba Chinese word segmentation algorithm.
  • kagome - Japanese morphological analyzer written in pure Go.

  • go-propisyu - Converts numbers to Russian words with correct grammatical gender and noun declension.

  • Ruby | Back to Top

  • Kevin Dias's A collection of Natural Language Processing (NLP) Ruby libraries, tools and software

  • Practical Natural Language Processing done in Ruby

  • Rust | Back to Top

  • whatlang — Natural language recognition library based on trigrams
  • rust-bert - Ready-to-use NLP pipelines and Transformer-based models

  • snips-nlu-rs (archived — Snips was discontinued) - A production ready library for intent parsing

  • NLP++ - NLP++ Language | Back to Top

  • VSCode Language Extension - NLP++ Language Extension for VSCode
  • nlp-engine - NLP++ engine to run NLP++ code on Linux including a full English parser
  • VisualText - Homepage for the NLP++ Language

  • NLP++ Wiki - Wiki entry for the NLP++ language

  • Julia | Back to Top

  • CorpusLoaders - A variety of loaders for various NLP corpora
  • Languages - A package for working with human languages
  • TextAnalysis - Julia package for text analysis
  • TextModels - Neural Network based models for Natural Language Processing
  • WordTokenizers - High performance tokenizers for natural language processing and other related tasks
  • Word2Vec - Julia interface to word2vec

Services

NLP as API with higher level functionality such as NER, Topic tagging and so on | Back to Top

  • Wit-ai - Natural Language Interface for apps and devices
  • IBM Watson's Natural Language Understanding - API and Github demo
  • Amazon Comprehend - NLP and ML suite covers most common tasks like NER, tagging, and sentiment analysis
  • Google Cloud Natural Language API - Syntax Analysis, NER, Sentiment Analysis, and Content tagging in atleast 9 languages include English and Chinese (Simplified and Traditional).
  • ParallelDots - High level Text Analysis API Service ranging from Sentiment Analysis to Intent Analysis
  • Microsoft Cognitive Service
  • TextRazor
  • Rosette
  • Textalytic - Natural Language Processing in the Browser with sentiment analysis, named entity extraction, POS tagging, word frequencies, topic modeling, word clouds, and more
  • NLP Cloud - SpaCy NLP models (custom and pre-trained ones) served through a RESTful API for named entity recognition (NER), POS tagging, and more.
  • Cloudmersive - Unified and free NLP APIs that perform actions such as speech tagging, text rephrasing, language translation/detection, and sentence parsing

Annotation Tools

  • GATE - General Architecture and Text Engineering is 15+ years old, free and open source
  • Anafora is free and open source, web-based raw text annotation tool
  • brat - brat rapid annotation tool is an online environment for collaborative text annotation
  • doccano - doccano is free, open-source, and provides annotation features for text classification, sequence labeling and sequence to sequence
  • INCEpTION - A semantic annotation platform offering intelligent assistance and knowledge management
  • prodigy is an annotation tool powered by active learning, costs $
  • LightTag - Hosted and managed text annotation tool for teams, costs $
  • rstWeb - open source local or online tool for discourse tree annotations
  • GitDox - open source server annotation tool with GitHub version control and validation for XML data and collaborative spreadsheet grids
  • Datasaur support various NLP tasks for individual or teams, freemium based
  • Konfuzio - team-first hosted and on-prem text, image and PDF annotation tool powered by active learning, freemium based, costs $
  • UBIAI - Easy-to-use text annotation tool for teams with most comprehensive auto-annotation features. Supports NER, relations and document classification as well as OCR annotation for invoice labeling, costs $
  • Shoonya - Shoonya is free and open source data annotation platform with wide varities of organization and workspace level management system. Shoonya is data agnostic, can be used by teams to annotate data with various level of verification stages at scale.
  • Annotation Lab - Free End-to-End No-Code platform for text annotation and DL model training/tuning. Out-of-the-box support for Named Entity Recognition, Classification, Relation extraction and Assertion Status Spark NLP models. Unlimited support for users, teams, projects, documents. Not FOSS.
  • FLAT - FLAT is a web-based linguistic annotation environment based around the FoLiA format, a rich XML-based format for linguistic annotation. Free and open source.
  • Argilla - open-source platform for collecting human feedback, building NLP and LLM datasets, and curating preference data.
  • Label Studio - open-core multi-modal labeling platform; widely used for NLP labeling.
  • Potato - Free, open-source annotation tool covering 21+ task types (classification, span, coreference, entity linking, agent trace evaluation) with built-in MACE quality control, attention checks, AI-assisted labeling, and 300+ example tasks.

README 内容较长，此处已截断，完整内容请查看 GitHub 仓库。

文档抓取自 GitHub 仓库 README，版权归原作者所有；已过滤徽章等噪音并经安全消毒后展示。