Metadata-Version: 2.1
Name: datahub-postgres-lineage
Version: 0.1.2
Summary: Extract table lineage from Postgres views
Home-page: https://github.com/contiamo/datahub-postgres-lineage
License: Apache-2.0
Keywords: datahub,postgres,sql lineage
Author: Contiamo
Author-email: developers@contiamo.com
Requires-Python: >=3.8.1,<3.11
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Dist: acryl-datahub[sqlalchemy] (>=0.9.3.2,<0.10.0.0)
Requires-Dist: geoalchemy2 (>=0.12.5,<0.13.0)
Requires-Dist: psycopg2-binary (>=2.9.5,<3.0.0)
Requires-Dist: pydantic (>=1.10.2,<2.0.0)
Project-URL: Repository, https://github.com/contiamo/datahub-postgres-lineage
Description-Content-Type: text/markdown

# Datahub Postgres View Lineage


A ingestion source to generate lineage for views in a Postgres database.


## Quick Start

First install [Poetry](https://python-poetry.org/docs/#installation) and [task](https://taskfile.dev) and initialize the project

```sh
task setup
```

Now, start a database

```sh
task start wait sample-view
```

Now run the ingestion to the console
```sh
task run
```

When it is successful, the output should include

```sh
Source (datahub_postgres_lineage.ingestion.PostgresLineageSource) report:
{'events_produced': '1',
 'events_produced_per_sec': '26',
 'event_ids': ['urn:li:dataset:(urn:li:dataPlatform:postgres,cool_db.public.emails,PROD)-upstreamLineage'],
 'warnings': {},
 'failures': {},
 'filtered': ['public.names'],
 'start_time': '2022-12-20 16:09:46.105046 (now).',
 'running_time': '0.04 seconds'}
```

## Configuration

| Key | Description | Default |
| --- | --- | --- |
| `username` | The username to connect to the database | '' |
| `password` | The password to connect to the database | '' |
| `host_port` | The host and port to connect to the database | '' |
| `database` | The database to connect to | '' |
| `database_alias` | Alias to apply to database when ingesting. | '' |
| `sqlalchemy_uri` | SQLAlchemy URI to connect to the database | '' |
| `scheme` | The SQLAlchemy scheme to use | `postgressql+psycopg2` |
| `schema_pattern` | | |
| `schema_pattern.allow`| Regexp pattern to match schemas to include | `.*` |
| `schema_pattern.deny` | Regexp pattern to match schemas to exclude, 'information_schema' and 'pg_catalog' are already excluded | '' |
| `view_pattern` | | |
| `view_pattern.allow` | Regexp pattern to match view names to include | `.*` |
| `view_pattern.deny` | Regexp pattern to match view names to exclude | '' |
