IntenseWebs/pgadmin4
3
0
Fork 0
mirror of https://github.com/pgadmin-org/pgadmin4.git synced 2024-11-25 10:10:19 -06:00
Code Issues Packages Projects Releases Wiki Activity

Add the ability to import and export server definitions from a config database. Fixes #3772

Browse Source
This commit is contained in:
Dave Page 2018-11-21 16:09:05 +00:00
parent 3cfc3366d7
commit f0327f5219
8 changed files with 606 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
docs/en_US/connecting.rst
View File

@ -4,7 +4,12 @@
`Connecting to a Server`:index:
*******************************
Before you can use the pgAdmin client to manage the objects that reside on your Postgres server, you must define a connection to the server. You can (optionally) use the *Server Group* dialog to create server groups to organize the server connections within the tree control for easier management. To open the *Server Group* dialog, right-click on the *Servers* node of the tree control, and select *Server Group* from the *Create* menu.
Before you can use the pgAdmin client to manage the objects that reside on your
Postgres server, you must define a connection to the server. You can
(optionally) use the *Server Group* dialog to create server groups to organize
the server connections within the tree control for easier management. To open
the *Server Group* dialog, right-click on the *Servers* node of the tree
control, and select *Server Group* from the *Create* menu.
Contents:
@ -12,7 +17,10 @@ Contents:
server_group_dialog
Use the fields on the *Server* dialog to define the connection properties for each new server that you wish to manage with pgAdmin. To open the *Server* dialog, right-click on the *Servers* node of the tree control, and select *Server* from the *Create* menu.
Use the fields on the *Server* dialog to define the connection properties for
each new server that you wish to manage with pgAdmin. To open the *Server*
dialog, right-click on the *Servers* node of the tree control, and select
*Server* from the *Create* menu.
Contents:
@ -20,11 +28,21 @@ Contents:
server_dialog
After defining a server connection, right-click on the server name, and select *Connect to server* to authenticate with the server, and start using pgAdmin to manage objects that reside on the server.
After defining a server connection, right-click on the server name, and select
*Connect to server* to authenticate with the server, and start using pgAdmin to
manage objects that reside on the server.
Contents:
.. toctree::
connect_to_server
connect_error
connect_error
Server definitions (and their groups) can be exported to a JSON file and
re-imported to the same or a different system to enable easy pre-configuration
of pgAdmin.
.. toctree::
export_import_servers

117
docs/en_US/export_import_servers.rst Normal file
View File

@ -0,0 +1,117 @@
.. _export_import_servers:
****************************************
`Exporting and importing Servers`:index:
****************************************
Server definitions (and their groups) can be exported to a JSON file and
re-imported to the same or a different system to enable easy pre-configuration
of pgAdmin. The ``setup.py`` script is used for this purpose.
.. note:: To export or import servers, you must use the Python interpreter that
is normally used to run pgAdmin to ensure that the required Python
packages are available. In most packages, this can be found in the
Python Virtual Environment that can be found in the installation
directory. When using platform-native packages, the system installation
of Python may be the one used by pgAdmin.
**Exporting Servers**
To export the servers defined in an installation, simply invoke ``setup.py`` with
the ``--dump-servers`` command line option, followed by the name (and if required,
path) to the desired output file. By default, servers owned by the desktop mode
user will be dumped (pgadmin4@pgadmin.org by default - see the DESKTOP_USER
setting in ``config.py``). This can be overridden with the ``--user`` command
line option. For example:
.. code-block:: bash
/path/to/python /path/to/setup.py --dump-servers output_file.json
# or, to specify a non-default user name:
/path/to/python /path/to/setup.py --dump-servers output_file.json --user user@example.com
To export only certain servers, use the ``--servers`` option and list one or
more server IDs. For example:
.. code-block:: bash
/path/to/python /path/to/setup.py --dump-servers output_file.json --server 1 2 5
**Importing Servers**
To import the servers defined in a JSON file, simply invoke ``setup.py`` with
the ``--load-servers`` command line option, followed by the name (and if required,
path) of the JSON file containing the server definitions. Servers will be owned
by the desktop mode user (pgadmin4@pgadmin.org by default - see the DESKTOP_USER
setting in ``config.py``). This can be overridden with the ``--user`` command
line option. For example:
.. code-block:: bash
/path/to/python /path/to/setup.py --load-servers input_file.json
# or, to specify a non-default user name to own the new servers:
/path/to/python /path/to/setup.py --load-servers input_file.json --user user@example.com
If any Servers are defined with a Server Group that is not already present in
the configuration database, the required Group will be created.
**JSON format**
The JSON file format used when importing or exporting servers is quite
straightforward and simply contains a list of servers, with a number of
attributes. The following attributes are required to be present in every server
definition: Name, Group, Port, Username, SSLMode, MaintenanceDB and one of Host,
HostAddr or Service.
Password fields cannot be imported or exported.
The following example shows both a minimally defined and a fully defined server:
.. code-block:: python
{
"Servers": {
"1": {
"Name": "Minimally Defined Server",
"Group": "Server Group 1",
"Port": 5432,
"Username": "postgres",
"Host": "localhost",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"2": {
"Name: "Fully Defined Server",
"Group": "Server Group 2",
"Host": "host.domain.com",
"HostAddr": "192.168.1.2",
"Port": 5432,
"MaintenanceDB": "postgres",
"Username": "postgres",
"Role": "my_role_name",
"SSLMode": "require",
"Comment": "This server has every option configured in the JSON",
"DBRestriction": "live_db test_db",
"PassFile": "/path/to/pgpassfile",
"SSLCert": "/path/to/sslcert.crt",
"SSLKey": "/path/to/sslcert.key",
"SSLRootCert": "/path/to/sslroot.crt",
"SSLCrl": "/path/to/sslcrl.crl",
"SSLCompression": 1,
"BGColor": "#ff9900",
"FGColor": "#000000",
"Service": "postgresql-10",
"Timeout": 60,
"UseSSHTunnel": 1,
"TunnelHost": "192.168.1.253",
"TunnelPort": 22,
"TunnelUsername": "username",
"TunnelAuthentication": 0
}
}
}

2
docs/en_US/release_notes_3_6.rst
View File

@ -11,6 +11,8 @@ Features
********
| `Feature #1513 <https://redmine.postgresql.org/issues/1513>`_ - Add support for dropping multiple objects at once from the collection Properties panel.
| `Feature #3772 <https://redmine.postgresql.org/issues/3772>`_ - Add the ability to import and export server definitions from a config database.
Bug fixes
*********

8
web/pgadmin/setup/tests/__init__.py Normal file
View File

@ -0,0 +1,8 @@
##########################################################################
#
# pgAdmin 4 - PostgreSQL Tools
#
# Copyright (C) 2013 - 2018, The pgAdmin Development Team
# This software is released under the PostgreSQL Licence
#
##########################################################################

41
web/pgadmin/setup/tests/servers.json Normal file
View File

@ -0,0 +1,41 @@
{
"Servers": {
"1": {
"Name": "Minimally Defined Server",
"Group": "Server Group 1",
"Port": 5432,
"Username": "postgres",
"Host": "localhost",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"2": {
"Name": "Fully Defined Server",
"Group": "Server Group 2",
"Host": "host.domain.com",
"HostAddr": "192.168.1.2",
"Port": 5432,
"MaintenanceDB": "postgres",
"Username": "postgres",
"Role": "my_role_name",
"SSLMode": "require",
"Comment": "This server has every option configured in the JSON",
"DBRestriction": "live_db test_db",
"PassFile": "/path/to/pgpassfile",
"SSLCert": "/path/to/sslcert.crt",
"SSLKey": "/path/to/sslcert.key",
"SSLRootCert": "/path/to/sslroot.crt",
"SSLCrl": "/path/to/sslcrl.crl",
"SSLCompression": 1,
"BGColor": "#ff9900",
"FGColor": "#000000",
"Service": "postgresql-10",
"Timeout": 60,
"UseSSHTunnel": 1,
"TunnelHost": "192.168.1.253",
"TunnelPort": "22",
"TunnelUsername": "username",
"TunnelAuthentication": 0
}
}
}

48
web/pgadmin/setup/tests/test_export_import_servers.py Normal file
View File

@ -0,0 +1,48 @@
##########################################################################
#
# pgAdmin 4 - PostgreSQL Tools
#
# Copyright (C) 2013 - 2018, The pgAdmin Development Team
# This software is released under the PostgreSQL Licence
#
##########################################################################
from pgadmin.utils.route import BaseTestGenerator
import os
import json
import tempfile
class ImportExportServersTestCase(BaseTestGenerator):
"""
This class validates the import/export servers functionality
"""
scenarios = [
# Fetching the default url for server group node
('Check Server Import/Export', dict())
]
def runTest(self):
path = os.path.dirname(__file__)
setup = os.path.realpath(os.path.join(path, "../../../setup.py"))
# Load the servers
os.system("python %s --load-servers %s 2> %s" %
(setup, os.path.join(path, "servers.json"), os.devnull))
# And dump them again
tf = tempfile.NamedTemporaryFile(delete=False)
os.system("python %s --dump-servers %s 2> %s" %
(setup, tf.name, os.devnull))
# Compare the JSON files, ignoring servers that exist in our
# generated file but not the test config (they are likely
# auto-discovered)
src = json.loads(open(os.path.join(path, "servers.json")).read())
tgt = json.loads(open(tf.name).read())
for server in src["Servers"]:
a = json.dumps(src["Servers"][server], sort_keys=True)
b = json.dumps(tgt["Servers"][server], sort_keys=True)
self.assertTrue(a, b)

20
web/servers.json Normal file
View File

@ -0,0 +1,20 @@
{
"Servers": {
"1": {
"Name": "OpenStack PEM Server",
"Group": "Servers",
"Host": "172.16.253.234",
"Port": 5432,
"MaintenanceDB": "postgres",
"Username": "postgres",
"SSLMode": "prefer",
"SSLCompression": 1,
"BGColor": "#ffffff",
"FGColor": "#000000",
"Timeout": 0,
"UseSSHTunnel": 0,
"TunnelPort": "22",
"TunnelAuthentication": 0
}
}
}

358
web/setup.py
View File

@ -10,9 +10,12 @@
"""Perform the initial setup of the application, by creating the auth
and settings database."""
import argparse
import json
import os
import sys
from pgadmin.model import db, Version, SCHEMA_VERSION as CURRENT_SCHEMA_VERSION
from pgadmin.model import db, User, Version, ServerGroup, Server, \
SCHEMA_VERSION as CURRENT_SCHEMA_VERSION
if sys.version_info[0] >= 3:
import builtins
@ -33,16 +36,311 @@ if sys.path[0] != root:
from pgadmin import create_app
if __name__ == '__main__':
# Configuration settings
import config
from pgadmin.model import SCHEMA_VERSION
from pgadmin.setup import db_upgrade, create_app_data_directory
config.SETTINGS_SCHEMA_VERSION = SCHEMA_VERSION
if "PGADMIN_TESTING_MODE" in os. environ and \
os.environ["PGADMIN_TESTING_MODE"] == "1":
config.SQLITE_PATH = config.TEST_SQLITE_PATH
def add_value(attr_dict, key, value):
"""Add a value to the attribute dict if non-empty.
Args:
attr_dict (dict): The dictionary to add the values to
key (str): The key for the new value
value (str): The value to add
Returns:
The updated attribute dictionary
"""
if value != "" and value is not None:
attr_dict[key] = value
return attr_dict
def dump_servers(args):
"""Dump the server groups and servers.
Args:
args (ArgParser): The parsed command line options
"""
app = create_app()
with app.app_context():
# What user?
if args.user is not None:
dump_user = args.user
else:
dump_user = config.DESKTOP_USER
user = User.query.filter_by(email=dump_user).first()
if user is None:
print("The specified user ID (%s) could not be found." %
dump_user)
sys.exit(1)
user_id = user.id
# Dict to collect the output
object_dict = {}
# Counters
servers_dumped = 0
# Dump servers
servers = Server.query.filter_by(user_id=user_id).all()
server_dict = {}
for server in servers:
if args.servers is None or str(server.id) in args.servers:
# Get the group name
group_name = ServerGroup.query.filter_by(
user_id=user_id, id=server.servergroup_id).first().name
attr_dict = {}
add_value(attr_dict, "Name", server.name)
add_value(attr_dict, "Group", group_name)
add_value(attr_dict, "Host", server.host)
add_value(attr_dict, "HostAddr", server.hostaddr)
add_value(attr_dict, "Port", server.port)
add_value(attr_dict, "MaintenanceDB", server.maintenance_db)
add_value(attr_dict, "Username", server.username)
add_value(attr_dict, "Role", server.role)
add_value(attr_dict, "SSLMode", server.ssl_mode)
add_value(attr_dict, "Comment", server.comment)
add_value(attr_dict, "DBRestriction", server.db_res)
add_value(attr_dict, "PassFile", server.passfile)
add_value(attr_dict, "SSLCert", server.sslcert)
add_value(attr_dict, "SSLKey", server.sslkey)
add_value(attr_dict, "SSLRootCert", server.sslrootcert)
add_value(attr_dict, "SSLCrl", server.sslcrl)
add_value(attr_dict, "SSLCompression", server.sslcompression)
add_value(attr_dict, "BGColor", server.bgcolor)
add_value(attr_dict, "FGColor", server.fgcolor)
add_value(attr_dict, "Service", server.service)
add_value(attr_dict, "Timeout", server.connect_timeout)
add_value(attr_dict, "UseSSHTunnel", server.use_ssh_tunnel)
add_value(attr_dict, "TunnelHost", server.tunnel_host)
add_value(attr_dict, "TunnelPort", server.tunnel_port)
add_value(attr_dict, "TunnelUsername", server.tunnel_username)
add_value(attr_dict, "TunnelAuthentication",
server.tunnel_authentication)
servers_dumped = servers_dumped + 1
server_dict[servers_dumped] = attr_dict
object_dict["Servers"] = server_dict
try:
f = open(args.dump_servers, "w")
except Exception as e:
print("Error opening output file %s: [%d] %s" %
(args.dump_servers, e.errno, e.strerror))
sys.exit(1)
try:
f.write(json.dumps(object_dict, indent=4))
except Exception as e:
print("Error writing output file %s: [%d] %s" %
(args.dump_servers, e.errno, e.strerror))
sys.exit(1)
f.close()
print("Configuration for %s servers dumped to %s." %
(servers_dumped, args.dump_servers))
def load_servers(args):
"""Load server groups and servers.
Args:
args (ArgParser): The parsed command line options
"""
try:
with open(args.load_servers) as f:
data = json.load(f)
except json.decoder.JSONDecodeError as e:
print("Error parsing input file %s: %s" %
(args.load_servers, e))
sys.exit(1)
except Exception as e:
print("Error reading input file %s: [%d] %s" %
(args.load_servers, e.errno, e.strerror))
sys.exit(1)
f.close()
app = create_app()
with app.app_context():
# What user?
if args.user is not None:
dump_user = args.user
else:
dump_user = config.DESKTOP_USER
user = User.query.filter_by(email=dump_user).first()
if user is None:
print("The specified user ID (%s) could not be found." %
dump_user)
sys.exit(1)
user_id = user.id
# Counters
groups_added = 0
servers_added = 0
# Get the server group
groups = ServerGroup.query.all()
def print_summary():
print("Added %d Server Group(s) and %d Server(s)." %
(groups_added, servers_added))
# Loop through the servers...
if "Servers" not in data:
print("'Servers' attribute not found in file '%s'" %
args.load_servers)
print_summary()
sys.exit(1)
for server in data["Servers"]:
obj = data["Servers"][server]
def check_attrib(attrib):
if attrib not in obj:
print("'%s' attribute not found for server '%s'" %
(attrib, server))
print_summary()
sys.exit(1)
check_attrib("Name")
check_attrib("Group")
check_attrib("Port")
check_attrib("Username")
check_attrib("SSLMode")
check_attrib("MaintenanceDB")
if "Host" not in obj and \
"HostAddr" not in obj and \
"Service" not in obj:
print("'Host', 'HostAddr' or 'Service' attribute not found "
"for server '%s'" % server)
print_summary()
sys.exit(1)
# Get the group. Create if necessary
group_id = -1
for g in groups:
if g.name == obj["Group"]:
group_id = g.id
break
if group_id == -1:
new_group = ServerGroup()
new_group.name = obj["Group"]
new_group.user_id = user_id
db.session.add(new_group)
try:
db.session.commit()
except Exception as e:
print("Error creating server group '%s': %s" %
(new_group.name, e))
print_summary()
sys.exit(1)
group_id = new_group.id
groups_added = groups_added + 1
groups = ServerGroup.query.all()
# Create the server
new_server = Server()
new_server.name = obj["Name"]
new_server.servergroup_id = group_id
new_server.user_id = user_id
new_server.host = obj["Host"]
if "HostAddr" in obj:
new_server.hostaddr = obj["HostAddr"]
new_server.port = obj["Port"]
new_server.maintenance_db = obj["MaintenanceDB"]
new_server.username = obj["Username"]
if "Role" in obj:
new_server.role = obj["Role"]
new_server.ssl_mode = obj["SSLMode"]
if "Comment" in obj:
new_server.comment = obj["Comment"]
if "DBRestriction" in obj:
new_server.db_res = obj["DBRestriction"]
if "PassFile" in obj:
new_server.passfile = obj["PassFile"]
if "SSLCert" in obj:
new_server.sslcert = obj["SSLCert"]
if "SSLKey" in obj:
new_server.sslkey = obj["SSLKey"]
if "SSLRootCert" in obj:
new_server.sslrootcert = obj["SSLRootCert"]
if "SSLCrl" in obj:
new_server.sslcrl = obj["SSLCrl"]
if "SSLCompression" in obj:
new_server.sslcompression = obj["SSLCompression"]
if "BGColor" in obj:
new_server.bgcolor = obj["BGColor"]
if "FGColor" in obj:
new_server.fgcolor = obj["FGColor"]
if "Service" in obj:
new_server.service = obj["Service"]
if "Timeout" in obj:
new_server.connect_timeout = obj["Timeout"]
if "UseSSHTunnel" in obj:
new_server.use_ssh_tunnel = obj["UseSSHTunnel"]
if "TunnelHost" in obj:
new_server.tunnel_host = obj["TunnelHost"]
if "TunnelPort" in obj:
new_server.tunnel_port = obj["TunnelPort"]
if "TunnelUsername" in obj:
new_server.tunnel_username = obj["TunnelUsername"]
if "TunnelAuthentication" in obj:
new_server.tunnel_authentication = obj["TunnelAuthentication"]
db.session.add(new_server)
try:
db.session.commit()
except Exception as e:
print("Error creating server '%s': %s" %
(new_server.name, e))
print_summary()
sys.exit(1)
servers_added = servers_added + 1
print_summary()
def setup_db():
"""Setup the configuration database."""
create_app_data_directory(config)
@ -70,3 +368,43 @@ if __name__ == '__main__':
version = Version.query.filter_by(name='ConfigDB').first()
version.value = CURRENT_SCHEMA_VERSION
db.session.commit()
if __name__ == '__main__':
# Configuration settings
import config
from pgadmin.model import SCHEMA_VERSION
from pgadmin.setup import db_upgrade, create_app_data_directory
parser = argparse.ArgumentParser(description='Setup the pgAdmin config DB')
imp_exp_group = parser.add_mutually_exclusive_group(required=False)
exp_group = imp_exp_group.add_argument_group('Dump server config')
exp_group.add_argument('--dump-servers', metavar="OUTPUT_FILE",
help='Dump the servers in the DB', required=False)
exp_group.add_argument('--servers', metavar="SERVERS", nargs='*',
help='One or more servers to dump', required=False)
imp_group = imp_exp_group.add_argument_group('Load server config')
imp_group.add_argument('--load-servers', metavar="INPUT_FILE",
help='Load servers into the DB', required=False)
imp_exp_group.add_argument('--user', metavar="USER_NAME",
help='Dump/load servers for the specified '
'username', required=False)
args, extra = parser.parse_known_args()
config.SETTINGS_SCHEMA_VERSION = SCHEMA_VERSION
if "PGADMIN_TESTING_MODE" in os.environ and \
os.environ["PGADMIN_TESTING_MODE"] == "1":
config.SQLITE_PATH = config.TEST_SQLITE_PATH
# What to do?
if args.dump_servers is not None:
dump_servers(args)
elif args.load_servers is not None:
load_servers(args)
else:
setup_db()