data-pipelines · GitHub Topics | GitHub 中文社区

#自然语言处理# RAGFlow 是一款基于深度文档理解构建的开源 RAG（Retrieval-Augmented Generation）引擎

document-understanding llm rag table-structure-recognition data-pipelines deep-learning document-parser nlp pdf-to-text preprocessing

Python23.51 k

6 小时前

dolphinscheduler

Apache软件基金会@apache

一个分布式易扩展的可视化DAG工作流任务调度系统。致力于解决数据处理流程中错综复杂的依赖关系，使调度系统在数据处理流程中开箱即用

workflow-schedule azkaban airflow task-scheduler job-scheduler cloud-native data-pipelines orchestration workflow workflow-orchestration

Java12.89 k

11 小时前🇨🇳

dagster

@dagster-io

An orchestration platform for the development, production, and observation of data assets.

翻译 - 用于构建数据应用程序的Python库：ETL，ML，数据管道等。

data-pipelines dagster workflow data-science workflow-automation python scheduler data-orchestrator etl analytics

Python11.79 k

1 小时前

unstructured

@Unstructured-IO

#自然语言处理# Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

deep-learning document-parsing machine-learning nlp ocr information-retrieval data-pipelines ml preprocessing pdf-to-text

HTML9.22 k

19 小时前

deeplake

@activeloopai

#数据仓库# Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop....

翻译 - 访问和管理PyTorch和TensorFlow数据集的最快方法。轻松构建可伸缩的数据管道。Leading Data 2.0 http://activeloop.ai

datasets data-pipelines deep-learning machine-learning data-science training pytorch tensorflow data-version-control collaboration

Python8.19 k

21 小时前

mage-ai

@mage-ai

#计算机科学# 🧙 Build, run, and manage data pipelines for integrating and transforming data.

machine-learning artificial-intelligence data data-engineering data-science python elt etl pipelines data-pipelines

Python7.98 k

19 小时前

elementary

@elementary-data

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

data-lineage data-governance data-warehouse snowflake bigquery data-analysis data-pipelines data-pipeline lineage data-reliability

HTML1.94 k

2 天前