Connection marked as broken because of SQLSTATE(08003)

Problem Statement: Customer is restarting PostgreSQL database every night. Long running jobs are failing from after DB restart.

https://github.com/brettwooldridge/HikariCP/issues/198

Address this:
https://github.com/brettwooldridge/HikariCP/issues/1056
Just to save the trouble of scanning through the PostgreSQL driver release notes, the Connection.setNetworkTimeout() was released in version 42.2.0 of the driver (https://jdbc.postgresql.org/documentation/changelog.html#version_42.2.0)

Available properties
https://stackoverflow.com/questions/26490967/how-do-i-configure-hikaricp-in-my-spring-boot-app-in-my-application-properties-f/51079239#51079239

Check when database was restarted
https://yongitz.wordpress.com/2013/12/05/getting-postgresql-servers-start-time-and-uptime/
To get the start time, execute the query below:
psql -c “SELECT pg_postmaster_start_time();”

To get the uptime, execute the query below:
psql -c “SELECT now() – pg_postmaster_start_time();”

Exception Handling for guaranteed write:

boolean writeStatus = false;
while(!writeStatus)
{
try{
write to database;
writeStatus = true;
}catch (Exception ex)
{
print exception;
sleep for 30 seconds;
//Hope for db to recover.
}
}

Advertisements

Dev, QA, Prod Monitoring Tools

Problem Statement: We need to monitor services on Dev, QA, and Prod.
Customer doesn’t want to sponsor the monitoring project.

Solution:
Step 1: Monitor & Alert
Monitor service endpoints using JMeter or Python.
Write output to CSV files day wise.
Send EMails or Slack messages for given thresholds

Step 2: Aggregate Numbers
Setup Jupyter Notebook
Mount all files as input with read-only access
Write a script to tabulate and visualize data.
Protect system with id/password
https://jupyter-notebook.readthedocs.io/en/stable/public_server.html

-o-

Python Data Analysis Library – Python

https://pandas.pydata.org/
pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

Pandas visualization
http://pandas.pydata.org/pandas-docs/stable/visualization.html
-o-

HDF5
Hierarchical Data Format – Version 5
https://support.hdfgroup.org/HDF5/whatishdf5.html
We can store/retrieve simple to complex data in flat files.
Community Version Vs Enterprise Edition: https://www.hdfgroup.org/solutions/hdf5-enterprise-support-edition/

-o-

Jupyter Note Book – Interactive way to run Python code

https://jupyter-notebook.readthedocs.io/en/stable/index.html

Deploy Jupyter in Cloud: https://kyso.io/

Sample csv file
https://catalog.data.gov/dataset/annual-crime-dataset-2015

Sample Python Note Book Code
https://kyso.io/polimetla/austin-crime
-o-