Added support for visualise the graph using a Line chart in the query tool. Fixes #7485

This commit is contained in:
Akshay Joshi 2022-06-22 17:18:51 +05:30
parent 41ceda01d0
commit 93bc1f3c57
38 changed files with 693 additions and 99 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 157 KiB

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 195 KiB

After

Width:  |  Height:  |  Size: 186 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 160 KiB

After

Width:  |  Height:  |  Size: 203 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 160 KiB

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 118 KiB

View File

@ -131,12 +131,25 @@ The Dashboards Node
Expand the *Dashboards* node to specify your dashboard display preferences.
.. image:: images/preferences_dashboard_graphs.png
:alt: Preferences dialog dashboard graph options
.. image:: images/preferences_dashboard_display.png
:alt: Preferences dialog dashboard display options
:align: center
Use the fields on the *Graphs* panel to specify your display preferences for
the graphs on the *Dashboard* tab:
* Set the warning and alert threshold value to highlight the long-running
queries on the dashboard.
* When the *Show activity?* switch is set to *True*, activity tables will be
displayed on dashboards.
* When the *Show graphs?* switch is set to *True*, graphs will be displayed on
dashboards.
.. image:: images/preferences_dashboard_refresh.png
:alt: Preferences dialog dashboard refresh options
:align: center
Use the fields on the *Refresh rates* panel to specify your refersh rates
preferences for the graphs on the *Dashboard* tab:
* Use the *Block I/O statistics refresh rate* field to specify the number of
seconds between block I/O statistic samples displayed in graphs.
@ -153,24 +166,6 @@ the graphs on the *Dashboard* tab:
* Use the *Tuples out refresh rate* field to specify the number of seconds
between tuples-out samples displayed in graphs.
.. image:: images/preferences_dashboard_display.png
:alt: Preferences dialog dashboard display options
:align: center
* Set the warning and alert threshold value to highlight the long-running
queries on the dashboard.
* When the *Show activity?* switch is set to *True*, activity tables will be
displayed on dashboards.
* When the *Show graph data points?* switch is set to *True*, data points will
be visible on graph lines.
* When the *Show graphs?* switch is set to *True*, graphs will be displayed on
dashboards.
* When the *Show mouse hover tooltip?* switch is set to *True*, a tooltip will
appear on mouse hover on the graph lines giving the data point details.
The Debugger Node
@ -197,6 +192,24 @@ ERD Tool window navigation:
:alt: Preferences dialog erd keyboard shortcuts section
:align: center
The Graphs Node
***************
Expand the *Graphs* node to specify your Graphs display preferences.
.. image:: images/preferences_dashboard_graphs.png
:alt: Preferences dialog dashboard graph options
:align: center
* When the *Show graph data points?* switch is set to *True*, data points will
be visible on graph lines.
* When the *Show mouse hover tooltip?* switch is set to *True*, a tooltip will
appear on mouse hover on the graph lines giving the data point details.
* When the *Use different data point styles?* switch is set to *True*,
data points will be visible in a different style on each graph lines.
The Miscellaneous Node
**********************

View File

@ -106,6 +106,45 @@ fully qualified with schema. Double quotes will be added if required.
For functions and procedures, the function name along with parameter names will
be pasted in the Query Tool.
Query History Panel
*******************
Use the *Query History* tab to review activity for the current session:
.. image:: images/query_output_history.png
:alt: Query tool history panel
:align: center
The Query History tab displays information about recent commands:
* The date and time that a query was invoked.
* The text of the query.
* The number of rows returned by the query.
* The amount of time it took the server to process the query and return a
result set.
* Messages returned by the server (not noted on the *Messages* tab).
* The source of the query (indicated by icons corresponding to the toolbar).
You can show or hide the queries generated internally by pgAdmin (during
'View/Edit Data' or 'Save Data' operations).
You can remove a single query by selecting it and clicking on the *Remove*
button. If you would like to remove all of the histories from the
*Query History* tab, then click on the *Remove All* button.
By using the *Copy* button, you can copy a particular query to the clipboard,
and with the *Copy to Query Editor* button, you can copy a specific query to
the Query Editor tab. During this operation, all existing content in the
Query Editor is erased.
Query History is maintained across sessions for each database on a per-user
basis when running in Query Tool mode. In View/Edit Data mode, history is not
retained. By default, the last 20 queries are stored for each database. This
can be adjusted in ``config_local.py`` or ``config_system.py`` (see the
:ref:`config.py <config_py>` documentation) by overriding the
`MAX_QUERY_HIST_STORED` value. See the :ref:`Deployment <deployment>` section
for more information.
The Data Output Panel
*********************
@ -288,44 +327,71 @@ particular channel.
:alt: Query tool notifications panel
:align: center
Query History Panel
*******************
Graph Visualiser Panel
**********************
Use the *Query History* tab to review activity for the current session:
Click the Graph Visualiser button in the toolbar to generate the *Graphs* of
the query results. The graph visualiser currently supports only Line Charts,
but more charts (Bar, Stacked Bar, Pie...) will be added soon.
.. image:: images/query_output_history.png
:alt: Query tool history panel
.. image:: images/query_graph_visualiser_panel.png
:alt: Query tool graph visualiser panel
:align: center
The Query History tab displays information about recent commands:
* X Axis
* The date and time that a query was invoked.
* The text of the query.
* The number of rows returned by the query.
* The amount of time it took the server to process the query and return a
result set.
* Messages returned by the server (not noted on the *Messages* tab).
* The source of the query (indicated by icons corresponding to the toolbar).
Choose the column whose value you wish to display on X-axis from the *X Axis*
dropdown. Select the *<Row Number>* option to use the number of rows as labels
on the X-axis.
You can show or hide the queries generated internally by pgAdmin (during
'View/Edit Data' or 'Save Data' operations).
.. image:: images/query_graph_xaxis.png
:alt: Query tool graph visualiser xaxis
:align: center
You can remove a single query by selecting it and clicking on the *Remove*
button. If you would like to remove all of the histories from the
*Query History* tab, then click on the *Remove All* button.
* Y Axis
By using the *Copy* button, you can copy a particular query to the clipboard,
and with the *Copy to Query Editor* button, you can copy a specific query to
the Query Editor tab. During this operation, all existing content in the
Query Editor is erased.
Choose the columns whose value you wish to display on Y-axis from the *Y Axis*
dropdown. Users can choose multiple columns. Choose the *<Select All>* option
from the drop-down menu to select all the columns.
Query History is maintained across sessions for each database on a per-user
basis when running in Query Tool mode. In View/Edit Data mode, history is not
retained. By default, the last 20 queries are stored for each database. This
can be adjusted in ``config_local.py`` or ``config_system.py`` (see the
:ref:`config.py <config_py>` documentation) by overriding the
`MAX_QUERY_HIST_STORED` value. See the :ref:`Deployment <deployment>` section
for more information.
.. image:: images/query_graph_yaxis.png
:alt: Query tool graph visualiser yaxis
:align: center
* Graph Type
Choose the type of the graph that you would like to generate. Currently only
*Line Charts* option is there, but more charts will be added soon.
.. image:: images/query_graph_type.png
:alt: Query tool graph visualiser graph type
:align: center
* Download and Zoom button
Zooming is performed by clicking and selecting an area over the chart with the
mouse. The *Zoom to original* button will bring you back to the original zoom
level.
Click the *Download* button on the button bar to download the chart.
.. image:: images/query_graph_toolbar.png
:alt: Query tool graph visualiser toolbar
:align: center
Line Chart
==========
The *Line Chart* can be generated by selecting the X-axis and the Y-axis and
clicking on the 'Generate' button. Below is an example of a chart of employee
names and their salaries.
.. image:: images/query_line_chart.png
:alt: Query tool graph visualiser line chart
:align: center
Set *Use different data point styles?* option to true in the :ref:`preferences`,
to show data points in a different style on each graph lines.
Connection Status
*****************
@ -382,6 +448,6 @@ The server will prompt you for confirmation to delete the macro.
To execute a macro, simply select the appropriate shortcut keys, or select it from the *Macros* menu.
.. image:: images/query_tool_macros_execution.png
.. image:: images/query_output_data.png
:alt: Query Tool Macros Execution
:align: center

View File

@ -191,6 +191,8 @@ Data Editing Options
| | a query has been executed and there are results in the data grid. You can specify the CSV/TXT | |
| | settings in the Preference Dialogue under SQL Editor -> CSV/TXT output. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+
| Graph Visualiser | Use the Graph Visualiser button to generate graphs of the query results. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+
Status Bar
**********

View File

@ -14,6 +14,7 @@ New features
| `Issue #7178 <https://redmine.postgresql.org/issues/7178>`_ - Added capability to deploy PostgreSQL servers on Microsoft Azure.
| `Issue #7332 <https://redmine.postgresql.org/issues/7332>`_ - Added support for passing password using Docker Secret to Docker images.
| `Issue #7351 <https://redmine.postgresql.org/issues/7351>`_ - Added the option 'Show template databases?' to display template databases regardless of the setting of 'Show system objects?'.
| `Issue #7485 <https://redmine.postgresql.org/issues/7485>`_ - Added support for visualise the graph using a Line chart in the query tool.
Housekeeping
************

View File

@ -106,6 +106,7 @@
"brace": "^0.11.1",
"browserfs": "^1.4.3",
"chart.js": "^3.0.0",
"chartjs-plugin-zoom": "^1.2.1",
"classnames": "^2.2.6",
"closest": "^0.0.1",
"codemirror": "^5.59.2",

View File

@ -131,7 +131,7 @@ define(
);
}
// Rerender the dashboard panel when preference value 'show graph' gets changed.
// Re-render the dashboard panel when preference value 'show graph' gets changed.
pgBrowser.onPreferencesChange('dashboards', function() {
getPanelView(
pgBrowser.tree,
@ -141,13 +141,23 @@ define(
);
});
// Re-render the dashboard panel when preference value gets changed.
pgBrowser.onPreferencesChange('graphs', function() {
getPanelView(
pgBrowser.tree,
$container[0],
pgBrowser,
myPanel._type
);
});
_.each([wcDocker.EVENT.CLOSED, wcDocker.EVENT.VISIBILITY_CHANGED],
function(ev) {
myPanel.on(ev, that.handleVisibility.bind(myPanel, ev));
});
pgBrowser.Events.on('pgadmin-browser:tree:selected', () => {
if(myPanel.isVisible() && myPanel._type !== 'properties') {
getPanelView(
pgBrowser.tree,

View File

@ -27,13 +27,16 @@ export function getPanelView(
panelType
) {
let item = !_.isNull(tree)? tree.selected(): null,
nodeData, node, treeNodeInfo, preferences, graphPref, dashPref;
nodeData, node, treeNodeInfo, preferences;
if (item){
nodeData = tree.itemData(item);
node = nodeData && pgBrowser.Nodes[nodeData._type];
treeNodeInfo = pgBrowser.tree.getTreeNodeHierarchy(item);
preferences = pgBrowser.get_preferences_for_module('dashboards');
dashPref = pgBrowser.get_preferences_for_module('dashboards');
graphPref = pgBrowser.get_preferences_for_module('graphs');
preferences = _.merge(dashPref, graphPref);
}
if (panelType == 'dashboard') {
ReactDOM.render(

View File

@ -21,7 +21,8 @@ from pgadmin.utils.ajax import precondition_required
from pgadmin.utils.driver import get_driver
from pgadmin.utils.menu import Panel
from pgadmin.utils.preferences import Preferences
from pgadmin.utils.constants import PREF_LABEL_DISPLAY, MIMETYPE_APP_JS
from pgadmin.utils.constants import PREF_LABEL_DISPLAY, MIMETYPE_APP_JS, \
PREF_LABEL_REFRESH_RATES
from config import PG_DEFAULT_DRIVER
@ -73,7 +74,7 @@ class DashboardModule(PgAdminModule):
"""
help_string = gettext('The number of seconds between graph samples.')
# Register options for the PG and PPAS help paths
# Register options for Dashboards
self.dashboard_preference = Preferences(
'dashboards', gettext('Dashboards')
)
@ -82,7 +83,7 @@ class DashboardModule(PgAdminModule):
'dashboards', 'session_stats_refresh',
gettext("Session statistics refresh rate"), 'integer',
1, min_val=1, max_val=999999,
category_label=gettext('Graphs'),
category_label=PREF_LABEL_REFRESH_RATES,
help_str=help_string
)
@ -90,7 +91,7 @@ class DashboardModule(PgAdminModule):
'dashboards', 'tps_stats_refresh',
gettext("Transaction throughput refresh rate"), 'integer',
1, min_val=1, max_val=999999,
category_label=gettext('Graphs'),
category_label=PREF_LABEL_REFRESH_RATES,
help_str=help_string
)
@ -98,7 +99,7 @@ class DashboardModule(PgAdminModule):
'dashboards', 'ti_stats_refresh',
gettext("Tuples in refresh rate"), 'integer',
1, min_val=1, max_val=999999,
category_label=gettext('Graphs'),
category_label=PREF_LABEL_REFRESH_RATES,
help_str=help_string
)
@ -106,7 +107,7 @@ class DashboardModule(PgAdminModule):
'dashboards', 'to_stats_refresh',
gettext("Tuples out refresh rate"), 'integer',
1, min_val=1, max_val=999999,
category_label=gettext('Graphs'),
category_label=PREF_LABEL_REFRESH_RATES,
help_str=help_string
)
@ -114,7 +115,7 @@ class DashboardModule(PgAdminModule):
'dashboards', 'bio_stats_refresh',
gettext("Block I/O statistics refresh rate"), 'integer',
1, min_val=1, max_val=999999,
category_label=gettext('Graphs'),
category_label=PREF_LABEL_REFRESH_RATES,
help_str=help_string
)
@ -134,23 +135,6 @@ class DashboardModule(PgAdminModule):
'will be displayed on dashboards.')
)
self.graph_data_points = self.dashboard_preference.register(
'display', 'graph_data_points',
gettext("Show graph data points?"), 'boolean', False,
category_label=PREF_LABEL_DISPLAY,
help_str=gettext('If set to True, data points will be '
'visible on graph lines.')
)
self.graph_mouse_track = self.dashboard_preference.register(
'display', 'graph_mouse_track',
gettext("Show mouse hover tooltip?"), 'boolean', True,
category_label=PREF_LABEL_DISPLAY,
help_str=gettext('If set to True, tooltip will appear on mouse '
'hover on the graph lines giving the data point '
'details')
)
self.long_running_query_threshold = self.dashboard_preference.register(
'display', 'long_running_query_threshold',
gettext('Long running query thresholds'), 'threshold',
@ -160,6 +144,36 @@ class DashboardModule(PgAdminModule):
'dashboard.')
)
# Register options for Graphs
self.graphs_preference = Preferences(
'graphs', gettext('Graphs')
)
self.graph_data_points = self.graphs_preference.register(
'graphs', 'graph_data_points',
gettext("Show graph data points?"), 'boolean', False,
category_label=PREF_LABEL_DISPLAY,
help_str=gettext('If set to True, data points will be '
'visible on graph lines.')
)
self.use_diff_point_style = self.graphs_preference.register(
'graphs', 'use_diff_point_style',
gettext("Use different data point styles?"), 'boolean', False,
category_label=PREF_LABEL_DISPLAY,
help_str=gettext('If set to True, data points will be visible '
'in a different style on each graph lines.')
)
self.graph_mouse_track = self.graphs_preference.register(
'graphs', 'graph_mouse_track',
gettext("Show mouse hover tooltip?"), 'boolean', True,
category_label=PREF_LABEL_DISPLAY,
help_str=gettext('If set to True, tooltip will appear on mouse '
'hover on the graph lines giving the data point '
'details')
)
def get_exposed_url_endpoints(self):
"""
Returns:

View File

@ -7,7 +7,7 @@
//
//////////////////////////////////////////////////////////////
import React, { useEffect, useRef, useState, useReducer, useCallback, useMemo } from 'react';
import {LineChart} from 'sources/chartjs';
import { LineChart, DATA_POINT_STYLE, DATA_POINT_SIZE } from 'sources/chartjs';
import {ChartContainer, DashboardRowCol, DashboardRow} from './Dashboard';
import url_for from 'sources/url_for';
import axios from 'axios';
@ -17,10 +17,9 @@ import {useInterval, usePrevious} from 'sources/custom_hooks';
import PropTypes from 'prop-types';
export const X_AXIS_LENGTH = 75;
export const POINT_SIZE = 2;
/* Transform the labels data to suit ChartJS */
export function transformData(labels, refreshRate) {
export function transformData(labels, refreshRate, use_diff_point_style) {
const colors = ['#00BCD4', '#9CCC65', '#E64A19'];
let datasets = Object.keys(labels).map((label, i)=>{
return {
@ -28,7 +27,8 @@ export function transformData(labels, refreshRate) {
data: labels[label] || [],
borderColor: colors[i],
backgroundColor: colors[i],
pointHitRadius: POINT_SIZE,
pointHitRadius: DATA_POINT_SIZE,
pointStyle: use_diff_point_style ? DATA_POINT_STYLE[i] : 'circle'
};
}) || [];
@ -225,11 +225,11 @@ export default function Graphs({preferences, sid, did, pageVisible, enablePoll=t
<div data-testid='graph-poll-delay' className='d-none'>{pollDelay}</div>
{chartDrawnOnce &&
<GraphsWrapper
sessionStats={transformData(sessionStats, preferences['session_stats_refresh'])}
tpsStats={transformData(tpsStats, preferences['tps_stats_refresh'])}
tiStats={transformData(tiStats, preferences['ti_stats_refresh'])}
toStats={transformData(toStats, preferences['to_stats_refresh'])}
bioStats={transformData(bioStats, preferences['bio_stats_refresh'])}
sessionStats={transformData(sessionStats, preferences['session_stats_refresh'], preferences['use_diff_point_style'])}
tpsStats={transformData(tpsStats, preferences['tps_stats_refresh'], preferences['use_diff_point_style'])}
tiStats={transformData(tiStats, preferences['ti_stats_refresh'], preferences['use_diff_point_style'])}
toStats={transformData(toStats, preferences['to_stats_refresh'], preferences['use_diff_point_style'])}
bioStats={transformData(bioStats, preferences['bio_stats_refresh'], preferences['use_diff_point_style'])}
errorMsg={errorMsg}
showTooltip={preferences['graph_mouse_track']}
showDataPoints={preferences['graph_data_points']}
@ -261,10 +261,9 @@ export function GraphsWrapper(props) {
const toStatsLegendRef = useRef();
const bioStatsLegendRef = useRef();
const options = useMemo(()=>({
normalized: true,
elements: {
point: {
radius: props.showDataPoints ? POINT_SIZE : 0,
radius: props.showDataPoints ? DATA_POINT_SIZE : 0,
},
},
plugins: {

View File

@ -8,12 +8,33 @@
//////////////////////////////////////////////////////////////
import React, { useEffect } from 'react';
import { Chart, registerables } from 'chart.js';
import zoomPlugin from 'chartjs-plugin-zoom';
import PropTypes from 'prop-types';
import _ from 'lodash';
export const DATA_POINT_STYLE = ['circle', 'cross', 'crossRot', 'rect',
'rectRounded', 'rectRot', 'star', 'triangle'];
export const DATA_POINT_SIZE = 3;
export const CHART_THEME_COLORS_LENGTH = 24;
export const CHART_THEME_COLORS = {
'standard':['#3366CC', '#DC3912', '#FF9900', '#109618', '#990099', '#0099C6',
'#DD4477', '#66AA00', '#B82E2E', '#316395', '#994499', '#22AA99',
'#AAAA11', '#6633CC', '#E67300', '#8B0707', '#651067', '#329262',
'#5574A6', '#3B3EAC', '#B77322', '#16D620', '#B91383', '#F4359E'],
'dark': ['#70E000', '#FF477E', '#7DC9F1', '#2EC4B6', '#52B788', '#2A9D8F',
'#E4E487', '#DB7C74', '#8AC926', '#979DAC', '#FF8FA3', '#7371FC', '#B388EB',
'#D4A276', '#FB5607', '#EEA236', '#FFEE32', '#EDC531', '#D4D700', '#FFFB69',
'#7FCC5C', '#50B0F0', '#3A86FF', '#00B4D8'],
'high_contrast': ['#00B4D8', '#2EC4B6', '#45D48A', '#50B0F0', '#52B788',
'#70E000', '#7DC9F1', '#7FCC5C', '#8AC926', '#979DAC', '#B388EB',
'#D4A276', '#D4D700', '#DEFF00', '#E4E487', '#EDC531', '#EEA236', '#F8845F',
'#FB4BF6', '#FF6C49', '#FF8FA3', '#FFEE32', '#FFFB69', '#FFFFFF']
};
const defaultOptions = {
responsive: true,
maintainAspectRatio: false,
normalized: true,
animation: {
duration: 0,
active: {
@ -41,6 +62,7 @@ const defaultOptions = {
},
ticks: {
display: false,
color: getComputedStyle(document.documentElement).getPropertyValue('--color-fg'),
},
},
y: {
@ -62,17 +84,20 @@ const defaultOptions = {
}
};
export default function BaseChart({type='line', id, options, data, redraw=false, ...props}) {
export default function BaseChart({type='line', id, options, data, redraw=false, plugins={}, ...props}) {
const chartRef = React.useRef();
const chartObj = React.useRef();
let optionsMerged = _.merge(defaultOptions, options);
const initChart = function() {
Chart.register(...registerables);
// Register for Zoom Plugin
Chart.register(zoomPlugin);
let chartContext = chartRef.current.getContext('2d');
chartObj.current = new Chart(chartContext, {
type: type,
data: data,
plugins: [plugins],
options: optionsMerged,
});
props.onInit && props.onInit(chartObj.current);
@ -123,6 +148,7 @@ BaseChart.propTypes = {
updateOptions: PropTypes.object,
onInit: PropTypes.func,
onUpdate: PropTypes.func,
plugins: PropTypes.object,
};
export function LineChart(props) {

View File

@ -879,7 +879,7 @@ export const InputSelect = forwardRef(({
const onChangeOption = useCallback((selectVal) => {
if (_.isArray(selectVal)) {
// Check if select all option is selected
if (!_.isUndefined(selectVal.find(x => x.label === 'Select All'))) {
if (!_.isUndefined(selectVal.find(x => x.label === '<Select All>'))) {
selectVal = filteredOptions;
}
/* If multi select options need to be in some format by UI, use formatter */
@ -905,7 +905,7 @@ export const InputSelect = forwardRef(({
openMenuOnClick: !readonly,
onChange: onChangeOption,
isLoading: isLoading,
options: controlProps.allowSelectAll ? [{ label: gettext('Select All'), value: '*' }, ...filteredOptions] : filteredOptions,
options: controlProps.allowSelectAll ? [{ label: gettext('<Select All>'), value: '*' }, ...filteredOptions] : filteredOptions,
value: realValue,
menuPortalTarget: document.body,
styles: styles,

View File

@ -105,6 +105,7 @@ class SqlEditorModule(PgAdminModule):
'sqleditor.poll',
'sqleditor.fetch',
'sqleditor.fetch_all',
'sqleditor.fetch_all_from_start',
'sqleditor.save',
'sqleditor.inclusive_filter',
'sqleditor.exclusive_filter',
@ -1087,6 +1088,49 @@ def fetch(trans_id, fetch_all=None):
)
@blueprint.route(
'/fetch_all_from_start/<int:trans_id>', methods=["GET"],
endpoint='fetch_all_from_start'
)
@login_required
def fetch_all_from_start(trans_id):
"""
This function is used to fetch all the records from start and reset
the cursor back to it's previous position.
"""
# Check the transaction and connection status
status, error_msg, conn, trans_obj, session_obj = \
check_transaction_status(trans_id)
if error_msg == ERROR_MSG_TRANS_ID_NOT_FOUND:
return make_json_response(success=0, errormsg=error_msg,
info='DATAGRID_TRANSACTION_REQUIRED',
status=404)
if status and conn is not None and session_obj is not None:
# Reset the cursor to start to fetch all the records.
conn.reset_cursor_at(0)
status, result = conn.async_fetchmany_2darray(-1)
if not status:
status = 'Error'
else:
status = 'Success'
# Reset the cursor back to it's actual position
conn.reset_cursor_at(trans_obj.get_fetched_row_cnt())
else:
status = 'NotConnected'
result = error_msg
return make_json_response(
data={
'status': status,
'result': result
}
)
def fetch_pg_types(columns_info, trans_obj):
"""
This method is used to fetch the pg types, which is required

View File

@ -397,6 +397,7 @@ export default class SQLEditor {
pgAdmin.Browser.preference_version(pgWindow.pgAdmin.Browser.preference_version());
pgAdmin.Browser.triggerPreferencesChange('browser');
pgAdmin.Browser.triggerPreferencesChange('sqleditor');
pgAdmin.Browser.triggerPreferencesChange('graphs');
}
}
}, 1000);

View File

@ -84,7 +84,7 @@ export default function QueryToolComponent({params, pgWindow, pgAdmin, selectedN
const containerRef = React.useRef(null);
const [qtState, _setQtState] = useState({
preferences: {
browser: {}, sqleditor: {},
browser: {}, sqleditor: {}, graphs: {}, misc: {},
},
is_new_tab: window.location == window.parent?.location,
current_file: null,
@ -217,6 +217,8 @@ export default function QueryToolComponent({params, pgWindow, pgAdmin, selectedN
setQtState({preferences: {
browser: pgWindow.pgAdmin.Browser.get_preferences_for_module('browser'),
sqleditor: pgWindow.pgAdmin.Browser.get_preferences_for_module('sqleditor'),
graphs: pgWindow.pgAdmin.Browser.get_preferences_for_module('graphs'),
misc: pgWindow.pgAdmin.Browser.get_preferences_for_module('misc'),
}});
}, []);
@ -311,6 +313,9 @@ export default function QueryToolComponent({params, pgWindow, pgAdmin, selectedN
pgWindow.pgAdmin.Browser.onPreferencesChange('sqleditor', function() {
reflectPreferences();
});
pgWindow.pgAdmin.Browser.onPreferencesChange('graphs', function() {
reflectPreferences();
});
/* WC docker events */
panel?.on(window.wcDocker.EVENT.CLOSING, function() {

View File

@ -68,6 +68,7 @@ export const QUERY_TOOL_EVENTS = {
RESET_LAYOUT: 'RESET_LAYOUT',
FORCE_CLOSE_PANEL: 'FORCE_CLOSE_PANEL',
RESET_GRAPH_VISUALISER: 'RESET_GRAPH_VISUALISER',
};
export const CONNECTION_STATUS = {
@ -95,4 +96,5 @@ export const PANELS = {
GEOMETRY: 'id-geometry',
NOTIFICATIONS: 'id-notifications',
HISTORY: 'id-history',
GRAPH_VISUALISER: 'id-graph-visualiser',
};

View File

@ -0,0 +1,343 @@
/////////////////////////////////////////////////////////////
//
// pgAdmin 4 - PostgreSQL Tools
//
// Copyright (C) 2013 - 2022, The pgAdmin Development Team
// This software is released under the PostgreSQL Licence
//
//////////////////////////////////////////////////////////////
import _ from 'lodash';
import React, { useContext, useState, useMemo, useEffect } from 'react';
import gettext from 'sources/gettext';
import Theme from 'sources/Theme';
import PropTypes from 'prop-types';
import url_for from 'sources/url_for';
import Loader from 'sources/components/Loader';
import { makeStyles } from '@material-ui/styles';
import { Box } from '@material-ui/core';
import ShowChartRoundedIcon from '@material-ui/icons/ShowChartRounded';
import ZoomOutMapIcon from '@material-ui/icons/ZoomOutMap';
import SaveAltIcon from '@material-ui/icons/SaveAlt';
import { InputSelect } from '../../../../../../static/js/components/FormComponents';
import { DefaultButton, PgButtonGroup, PgIconButton} from '../../../../../../static/js/components/Buttons';
import { LineChart, DATA_POINT_STYLE, DATA_POINT_SIZE,
CHART_THEME_COLORS, CHART_THEME_COLORS_LENGTH} from 'sources/chartjs';
import { QueryToolEventsContext, QueryToolContext } from '../QueryToolComponent';
import { QUERY_TOOL_EVENTS, PANELS } from '../QueryToolConstants';
import { LayoutHelper } from '../../../../../../static/js/helpers/Layout';
// Numeric data type used to separate out the options for Y axis.
const NUMERIC_TYPES = ['oid', 'smallint', 'integer', 'bigint', 'decimal', 'numeric',
'real', 'double precision', 'smallserial', 'serial', 'bigserial'];
const useStyles = makeStyles((theme)=>({
mainContainer: {
width: '100%',
height: '100%',
overflowY: 'scroll',
display: 'flex',
flexDirection: 'column',
},
topContainer: {
alignItems: 'flex-start',
padding: '4px',
backgroundColor: theme.otherVars.editorToolbarBg,
flexWrap: 'wrap',
...theme.mixins.panelBorder.bottom,
},
displayFlex: {
display: 'flex',
},
graphContainer: {
padding: '8px',
flexGrow: 1,
overflow: 'hidden',
},
spanLabel: {
alignSelf: 'center',
marginRight: '4px',
},
selectCtrl: {
minWidth: '200px',
},
}));
// This function is used to generate the appropriate graph based on the graphType.
function GenerateGraph({graphType, graphData, ...props}) {
const queryToolCtx = useContext(QueryToolContext);
let showDataPoints = queryToolCtx.preferences.graphs['graph_data_points'];
let useDiffPointStyle = queryToolCtx.preferences.graphs['use_diff_point_style'];
let showToolTip = queryToolCtx.preferences.graphs['graph_mouse_track'];
// Below options are used by chartjs while rendering the graph
const options = useMemo(()=>({
elements: {
point: {
radius: showDataPoints ? DATA_POINT_SIZE : 0,
},
},
plugins: {
legend: {
display: true,
labels: {
usePointStyle: (showDataPoints && useDiffPointStyle) ? true : false
},
},
tooltip: {
enabled: showToolTip,
},
zoom: {
pan: {
enabled: true,
mode: 'x',
modifierKey: 'ctrl',
},
zoom: {
drag: {
enabled: true,
borderColor: 'rgb(54, 162, 235)',
borderWidth: 1,
backgroundColor: 'rgba(54, 162, 235, 0.3)'
},
mode: 'xy',
},
}
},
scales: {
x: {
display: true,
ticks: {
display: true,
},
},
},
}));
if (_.isEmpty(graphData.datasets))
return null;
if (graphType == 'L') {
return <LineChart options={options} data={graphData} {...props}/>;
} else {
return null;
}
}
GenerateGraph.propTypes = {
graphType: PropTypes.string,
graphData: PropTypes.object,
};
// This function is used to get the data set for the X axis and Y axis
function getGraphDataSet(rows, columns, xaxis, yaxis, queryToolCtx) {
// Function is used to the find the position of the column
function getColumnPosition(colName) {
return _.find(columns, (c)=>(c.name==colName))?.pos;
}
let styleIndex = -1, colorIndex = -1;
return {
'labels': rows.map((r, index)=>{
let colPosition = getColumnPosition(xaxis);
// If row number are selected then label should be the index + 1.
if (xaxis === '<Row Number>') {
return index + 1;
}
return r[colPosition];
}),
'datasets': yaxis.map((colName)=>{
// Loop is used to set the index for random color array
if (colorIndex >= (CHART_THEME_COLORS_LENGTH - 1)) {
colorIndex = -1;
}
colorIndex = colorIndex + 1;
let color = CHART_THEME_COLORS[queryToolCtx.preferences.misc.theme][colorIndex];
let colPosition = getColumnPosition(colName);
// Loop is used to set the index for DATA_POINT_STYLE array
if (styleIndex >= (DATA_POINT_STYLE.length - 1)) {
styleIndex = -1;
}
styleIndex = styleIndex + 1;
return {
label: colName,
data: rows.map((r)=>r[colPosition]),
backgroundColor: color,
borderColor:color,
pointHitRadius: DATA_POINT_SIZE,
pointHoverRadius: 5,
pointStyle: queryToolCtx.preferences.graphs['use_diff_point_style'] ? DATA_POINT_STYLE[styleIndex] : 'circle'
};
}),
};
}
export function GraphVisualiser({initColumns}) {
const classes = useStyles();
const chartObjRef = React.useRef();
const contentRef = React.useRef();
const eventBus = useContext(QueryToolEventsContext);
const queryToolCtx = useContext(QueryToolContext);
const [graphType, setGraphType] = useState('L');
const [xaxis, setXAxis] = useState(null);
const [yaxis, setYAxis] = useState([]);
const [graphData, setGraphData] = useState({'datasets': []});
const [loaderText, setLoaderText] = useState('');
const [columns, setColumns] = useState(initColumns);
const [graphHeight, setGraphHeight] = useState();
// Create X axis options for drop down.
let xAxisOptions = useMemo(()=>{
let retVal = [{label:gettext('<Row Number>'), value:'<Row Number>'}];
columns.forEach((element) => {
if (!element.is_array) {
retVal.push({label:gettext(element.name), value:element.name});
}
});
return retVal;
}, [columns]);
// Create Y axis options for drop down which must be of numeric type.
let yAxisOptions = useMemo(()=>{
let retVal = [];
columns.forEach((element) => {
if (!element.is_array && NUMERIC_TYPES.indexOf(element.type) >= 0) {
retVal.push({label:gettext(element.name), value:element.name});
}
});
return retVal;
}, [columns]);
// optionsReload is required to reset the X axis and Y axis option in InputSelect.
let optionsReload = useMemo(()=>{
return columns.map((c)=>c.name).join('');
}, [columns]);
// Use to register/deregister query execution end event. We need to reset graph
// when query is changed and the execution of the query is ended.
useEffect(()=>{
let timeoutId;
const contentResizeObserver = new ResizeObserver(()=>{
clearTimeout(timeoutId);
if(LayoutHelper.isTabVisible(queryToolCtx.docker, PANELS.GRAPH_VISUALISER)) {
timeoutId = setTimeout(function () {
setGraphHeight(contentRef.current.offsetHeight);
}, 300);
}
});
contentResizeObserver.observe(contentRef.current);
const resetGraphVisualiser = (newColumns)=>{
setGraphData({'datasets': []});
setXAxis(null);
setYAxis([]);
setColumns(newColumns);
};
eventBus.registerListener(QUERY_TOOL_EVENTS.RESET_GRAPH_VISUALISER, resetGraphVisualiser);
return ()=>{
eventBus.deregisterListener(QUERY_TOOL_EVENTS.RESET_GRAPH_VISUALISER, resetGraphVisualiser);
};
}, []);
// Generate button callback
const onGenerate = async ()=>{
setLoaderText(gettext('Fetching all the records...'));
let url = url_for('sqleditor.fetch_all_from_start', {
'trans_id': queryToolCtx.params.trans_id
});
let res = await queryToolCtx.api.get(url);
// Set the Graph Data
setGraphData(
getGraphDataSet(res.data.data.result, columns, xaxis, yaxis, queryToolCtx)
);
setLoaderText('');
};
// Reset the zoom to normal
const onResetZoom = ()=> {
chartObjRef.current.resetZoom();
};
// Download button callback
const onDownloadGraph = ()=> {
let a = document.createElement('a');
a.href = chartObjRef.current.toBase64Image();
a.download = 'graph_visualiser-' + new Date().getTime() + '.png';
a.click();
};
// This plugin is used to set the background color of the canvas. Very useful
// when downloading the graph.
const plugin = {
beforeDraw: (chart) => {
const ctx = chart.canvas.getContext('2d');
ctx.save();
ctx.globalCompositeOperation = 'destination-over';
ctx.fillStyle = getComputedStyle(document.documentElement).getPropertyValue('--color-bg');
ctx.fillRect(0, 0, chart.width, chart.height);
ctx.restore();
}
};
return (
<Theme>
<Box className={classes.mainContainer}>
<Loader message={loaderText} />
<Box className={classes.topContainer}>
<Box className={classes.displayFlex}>
<Box className={classes.displayFlex}>
<span className={classes.spanLabel}>{gettext('X Axis')}</span>
<InputSelect className={classes.selectCtrl} options={xAxisOptions}
onChange={(v)=>setXAxis(v)} value={xaxis} optionsReloadBasis={optionsReload}/>
</Box>
<Box className={classes.displayFlex} marginLeft="auto">
<span className={classes.spanLabel} >{gettext('Graph Type')}</span>
<InputSelect className={classes.selectCtrl} controlProps={{allowClear: false}}
options={[
{label: gettext('Line Chart'), value: 'L'},
{label: gettext('<More charts coming soon>'), value: 'L'}
]} onChange={(v)=>setGraphType(v)} value={graphType} />
</Box>
<DefaultButton onClick={onGenerate} startIcon={<ShowChartRoundedIcon />}
disabled={ _.isEmpty(xaxis) || yaxis.length <= 0 }>
{gettext('Generate')}
</DefaultButton>
</Box>
<Box className={classes.displayFlex}>
<span className={classes.spanLabel}>{gettext('Y Axis')}</span>
<InputSelect className={classes.selectCtrl} controlProps={{'multiple': true, allowSelectAll: true}}
options={yAxisOptions} onChange={(v)=>setYAxis(v)} value={yaxis} optionsReloadBasis={optionsReload}/>
</Box>
</Box>
<Box display="flex" marginLeft="3px" marginTop="3px">
<PgButtonGroup size="small">
<PgIconButton title={gettext('Zoom to original')} icon={<ZoomOutMapIcon style={{height: '1.2rem'}}/>}
onClick={()=>onResetZoom()} disabled={ graphData.datasets.length <= 0 }/>
<PgIconButton title={gettext('Download')} icon={<SaveAltIcon style={{height: '1.2rem'}}/>}
onClick={onDownloadGraph} disabled={ graphData.datasets.length <= 0 }/>
</PgButtonGroup>
</Box>
<Box ref={contentRef} className={classes.graphContainer}>
<Box style={{height:`${graphHeight}px`}}>
<GenerateGraph graphType={graphType} graphData={graphData} onInit={(chartObj)=> {
chartObjRef.current = chartObj;
}} plugins={plugin}/>
</Box>
</Box>
</Box>
</Theme>
);
}
GraphVisualiser.propTypes = {
initColumns: PropTypes.array
};

View File

@ -28,6 +28,7 @@ import moment from 'moment';
import ConfirmSaveContent from '../dialogs/ConfirmSaveContent';
import { makeStyles } from '@material-ui/styles';
import EmptyPanelMessage from '../../../../../../static/js/components/EmptyPanelMessage';
import { GraphVisualiser } from './GraphVisualiser';
export class ResultSetUtils {
constructor(api, transId, isQueryTool=true) {
@ -919,6 +920,10 @@ export function ResultSet() {
rsu.current.transId = queryToolCtx.params.trans_id;
}, [queryToolCtx.params.trans_id]);
useEffect(()=>{
eventBus.fireEvent(QUERY_TOOL_EVENTS.RESET_GRAPH_VISUALISER, columns);
}, [columns]);
const fetchMoreRows = async (all=false, callback)=>{
if(queryData.has_more_rows) {
setIsLoadingMore(true);
@ -1259,6 +1264,22 @@ export function ResultSet() {
setRows(newRows);
};
useEffect(()=>{
const showGraphVisualiser = async ()=>{
LayoutHelper.openTab(queryToolCtx.docker, {
id: PANELS.GRAPH_VISUALISER,
title: gettext('Graph Visualiser'),
content: <GraphVisualiser initColumns={columns} />,
closable: true,
}, PANELS.MESSAGES, 'after-tab', true);
};
eventBus.registerListener(QUERY_TOOL_EVENTS.TRIGGER_GRAPH_VISUALISER, showGraphVisualiser);
return ()=>{
eventBus.deregisterListener(QUERY_TOOL_EVENTS.TRIGGER_GRAPH_VISUALISER, showGraphVisualiser);
};
}, [queryToolCtx.docker, columns]);
const rowKeyGetter = React.useCallback((row)=>row[rsu.current.clientPK]);
return (
<Box className={classes.root} ref={containerRef} tabIndex="0">

View File

@ -14,6 +14,7 @@ import KeyboardArrowDownIcon from '@material-ui/icons/KeyboardArrowDown';
import PlaylistAddRoundedIcon from '@material-ui/icons/PlaylistAddRounded';
import FileCopyRoundedIcon from '@material-ui/icons/FileCopyRounded';
import DeleteRoundedIcon from '@material-ui/icons/DeleteRounded';
import TimelineRoundedIcon from '@material-ui/icons/TimelineRounded';
import { PasteIcon, SaveDataIcon } from '../../../../../../static/js/components/ExternalIcon';
import GetAppRoundedIcon from '@material-ui/icons/GetAppRounded';
import {QUERY_TOOL_EVENTS} from '../QueryToolConstants';
@ -82,6 +83,9 @@ export function ResultSetToolbar({containerRef, canEdit, totalRowCount}) {
const downloadResult = useCallback(()=>{
eventBus.fireEvent(QUERY_TOOL_EVENTS.TRIGGER_SAVE_RESULTS);
}, []);
const showGraphVisualiser = useCallback(()=>{
eventBus.fireEvent(QUERY_TOOL_EVENTS.TRIGGER_GRAPH_VISUALISER);
}, []);
const openMenu = useCallback((e)=>{
setMenuOpenId(e.currentTarget.name);
@ -158,6 +162,10 @@ export function ResultSetToolbar({containerRef, canEdit, totalRowCount}) {
onClick={downloadResult} shortcut={queryToolPref.download_results}
disabled={buttonsDisabled['save-result']} />
</PgButtonGroup>
<PgButtonGroup size="small">
<PgIconButton title={gettext('Graph Visualiser')} icon={<TimelineRoundedIcon />}
onClick={showGraphVisualiser} disabled={buttonsDisabled['save-result']} />
</PgButtonGroup>
</Box>
<PgMenu
anchorRef={copyMenuRef}

View File

@ -25,6 +25,7 @@ PREF_LABEL_CSV_TXT = gettext('CSV/TXT Output')
PREF_LABEL_RESULTS_GRID = gettext('Results grid')
PREF_LABEL_SQL_FORMATTING = gettext('SQL formatting')
PREF_LABEL_TABS_SETTINGS = gettext('Tab settings')
PREF_LABEL_REFRESH_RATES = gettext('Refresh rates')
PGADMIN_STRING_SEPARATOR = '_$PGADMIN$_'
PGADMIN_NODE = 'pgadmin.node.%s'

View File

@ -723,6 +723,25 @@ WHERE db.datname = current_database()""")
return True, cur
def reset_cursor_at(self, position):
"""
This function is used to reset the cursor at the given position
"""
cur = self.__async_cursor
if not cur:
current_app.logger.log(
25,
'Cursor not found in reset_cursor_at method')
try:
cur.scroll(position, mode='absolute')
except psycopg2.Error:
# bypassing the error as cursor tried to scroll on the
# specified position, but end of records found
current_app.logger.log(
25,
'Failed to reset cursor in reset_cursor_at method')
def escape_params_sqlascii(self, params):
# The data is unescaped using string_typecasters when selected
# We need to esacpe the data so that it does not fail when

View File

@ -2,26 +2,29 @@ import jasmineEnzyme from 'jasmine-enzyme';
import React from 'react';
import {mount} from 'enzyme';
import '../helper/enzyme.helper';
import { DATA_POINT_SIZE } from 'sources/chartjs';
import Graphs, {GraphsWrapper, X_AXIS_LENGTH, POINT_SIZE, transformData,
import Graphs, {GraphsWrapper, X_AXIS_LENGTH, transformData,
getStatsUrl, statsReducer} from '../../../pgadmin/dashboard/static/js/Graphs';
describe('Graphs.js', ()=>{
it('transformData', ()=>{
expect(transformData({'Label1': [], 'Label2': []}, 1)).toEqual({
expect(transformData({'Label1': [], 'Label2': []}, 1, false)).toEqual({
labels: [...Array(X_AXIS_LENGTH).keys()],
datasets: [{
label: 'Label1',
data: [],
borderColor: '#00BCD4',
backgroundColor: '#00BCD4',
pointHitRadius: POINT_SIZE,
pointHitRadius: DATA_POINT_SIZE,
pointStyle: 'circle',
},{
label: 'Label2',
data: [],
borderColor: '#9CCC65',
backgroundColor: '#9CCC65',
pointHitRadius: POINT_SIZE,
pointHitRadius: DATA_POINT_SIZE,
pointStyle: 'circle',
}],
refreshRate: 1,
});

View File

@ -3944,6 +3944,13 @@ chart.js@^3.0.0:
resolved "https://registry.yarnpkg.com/chart.js/-/chart.js-3.8.0.tgz#c6c14c457b9dc3ce7f1514a59e9b262afd6f1a94"
integrity sha512-cr8xhrXjLIXVLOBZPkBZVF6NDeiVIrPLHcMhnON7UufudL+CNeRrD+wpYanswlm8NpudMdrt3CHoLMQMxJhHRg==
chartjs-plugin-zoom@^1.2.1:
version "1.2.1"
resolved "https://registry.yarnpkg.com/chartjs-plugin-zoom/-/chartjs-plugin-zoom-1.2.1.tgz#7e350ba20d907f397d0c055239dcc67d326df705"
integrity sha512-2zbWvw2pljrtMLMXkKw1uxYzAne5PtjJiOZftcut4Lo3Ee8qUt95RpMKDWrZ+pBZxZKQKOD/etdU4pN2jxZUmg==
dependencies:
hammerjs "^2.0.8"
cheerio-select@^1.5.0:
version "1.5.0"
resolved "https://registry.yarnpkg.com/cheerio-select/-/cheerio-select-1.5.0.tgz#faf3daeb31b17c5e1a9dabcee288aaf8aafa5823"
@ -5646,6 +5653,11 @@ gzip-size@^6.0.0:
dependencies:
duplexer "^0.1.2"
hammerjs@^2.0.8:
version "2.0.8"
resolved "https://registry.yarnpkg.com/hammerjs/-/hammerjs-2.0.8.tgz#04ef77862cff2bb79d30f7692095930222bf60f1"
integrity sha512-tSQXBXS/MWQOn/RKckawJ61vvsDpCom87JgxiYdGwHdOa0ht0vzUWDlfioofFCRU0L+6NGDt6XzbgoJvZkMeRQ==
handlebars@^4.0.11:
version "4.7.7"
resolved "https://registry.yarnpkg.com/handlebars/-/handlebars-4.7.7.tgz#9ce33416aad02dbd6c8fafa8240d5d98004945a1"