Export all airflow connections to new environment

邮差的信 提交于 2020-01-15 06:22:05

问题


I'm trying to migrate all the existing airflow connections to a new airflow.

I was looking at the cli options airflow connections --help, it gives an option to list but doesn't give an option to export/import to/from json format.

Is there a way via cli/airflow ui to migrate connections across multiple airflows?


回答1:


You can either connect directly to the Airflow meta db and dump those connections, then load them in a separate database. However, if you want to automate something like this, you can get started by dumping them in a CSV file:

from airflow.utils import db
from airflow.models import Connection
import csv

outfile = open('myconnections.csv', 'w')
outcsv = csv.writer(outfile)

with db.create_session() as session:
    connections = session.query(Connection).all()

conn_list = [
    [getattr(c, column.name) for column in Connection.__mapper__.columns]
    for c in connections
]
outcsv.writerows(conn_list)
outfile.close()

After that, you can load that to a new DB manually or with a similar script.

IMPORTANT: if you have enabled encryption, the passwords stored for these connections will be encrypted, and when you load them to the new DB, you must use the identical fernet key, or otherwise you won't be able to decrypt them.



来源:https://stackoverflow.com/questions/55626195/export-all-airflow-connections-to-new-environment

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!