Category: dev-tools

PostgreSQL 13 and DBeaver CE on Debian

Install PostgreSQL 13 on a server:

sudo apt install postgresql-13

Create a PostgreSQL role and database for an existing OS user:

sudo -u postgres createuser --pwprompt oliver
sudo -u postgres createdb -O oliver oliver

Then make sure you have DBeaver CE installed on a client machine.

Create a connection from DBeaver to Postgres:

Configure authentication to database “oliver” via localhost, as we will use an ssh tunnel, and leave password empty:

Configure ssh tunnel, for example if the server host is named “tc”. In the example we rely on a running ssh agent, which was explained in an earlier blog post.

Then click “Connect” in the context menu of the new connection entry, and enter the password you assigned to the role:

And you should be connected:

Advertisement

Safely install DBeaver CE on Debian

To install DBeaver CE (community edition) on a client machine using apt, we can add their apt repository to our apt source, but it is not recommended to add 3rd party keys to the global apt keyring. That’s why the whole apt-key command is actually deprecated.

For more background info, see https://wiki.debian.org/DebianRepository/UseThirdParty

So let’s download the DBeaver key and prepare it for safe use:

curl https://dbeaver.io/debs/dbeaver.gpg.key | gpg --dearmor > dbeaver.gpg
sudo mkdir -p /usr/local/share/keyrings
sudo mv dbeaver.gpg /usr/local/share/keyrings
sudo chown root.root /usr/local/share/keyrings/dbeaver.gpg
sudo chmod 400 /usr/local/share/keyrings/dbeaver.gpg

Then add the DBeaver repo to your apt sources with the proper signed-by annotation:

echo "deb [signed-by=/usr/local/share/keyrings/dbeaver.gpg] https://dbeaver.io/debs/dbeaver-ce /" | sudo tee /etc/apt/sources.list.d/dbeaver.list

And finally we can install DBeaver CE:

sudo apt update
sudo apt install dbeaver-ce

If all went well, you should now be able to run DBeaver from the “Development” section of your Applications menu.

Undo/Redo in Java using Protostuff serialization and binary diffs

Many applications need Undo/Redo functionality. Commonly used implementation patterns are:

  • Command Pattern
  • Memento Pattern (state snapshots)
  • State diffs

When using the Command Pattern one would encapsulate both the change logic and its reversal in command objects. Undo/Redo is implemented by managing stacks of those objects. This approach has its limitations, for example for changes that are unidirectional in nature, like anything involving randomness, encryption, etc.

State snapshots save the full state of the edited data as object graphs or some representation thereof. This is also called the Memento Pattern. It often uses serialization and typically compression of the object graph to reduce memory use and ensure immutable snapshots that can also be stored out-of-process, if desired.

State diffs are based on the idea of State snapshots, but only store the difference between states. This can vastly reduce memory consumption of your Undo/Redo history. It is based on diffing algorithms that compute the delta between two states (or their memento) and allow Undo/Redo by applying the deltas as patches against a given state. A disadvantage is that jumping to a state involves a whole chain of patch applications. But it is a good approach when the user mainly navigates the Undo/Redo history sequentially.

A highly reusable implementation of Undo/Redo using State Diffs is available at my github account: https://github.com/odoepner/diffing-history

It uses the following Open Source libraries:

  • Protostuff for object graph serialization using runtime schema
  • JavaxDelta for binary diffing and patching

It provides the following features:

  • Unlimited Undo and Redo
  • Can handle any type of Java objects
  • Low memory footprint
  • Straightforward type-safe API
  • Supports stack size listeners
  • Gzip compression for the serialized current state

It is Open Source under the Unlicense.

Usage

The main API is the History interface.
Create an instance of DiffingHistory to get started.
The DiffingHistoryTest calls all History methods and illustrates the API.

Recursively compare content of two directories

Command line

This requires the diff and vim packages.

diff --recursive /dir/ect/ory1 /dir/ect/ory2 > 1_vs_2.diff
vimdiff 1_vs_2.diff

Potentially useful diff options:

--ignore-all-space
--exclude=.svn

GUI

Install Intellij CE.

Then either Run IntelliJ Diff from the command-line.

Or from within a running Intellij window:

  • Open a common parent directory as a project
  • Select the two directories to compare
  • Right-click – Compare Directories

Alternatives

I often see the GPL-licensed WinMerge tool recommended, But it works only on Windows, last release was 2013 and navigation into sub-directories and file diffs is a bit clunkier than in Intellij.

Install portable JDK on Windows without admin rights

I found the basic idea here, the exact steps are:

iron-java-mug_120x120

  1. Install Portable 7zip
  2. Download Oracle JDK installer for Windows (*.exe)
  3. Run 7-ZipPortable.exe from your Portable 7zip
  4. In 7zip find and right-click the jdk installer exe file
  5. From the context menu use 7-Zip – Open Archive and then Extract
  6. Now extract the resulting “tools.zip” to a folder that is writable for you
  7. Open a cmd.exe, cd into the folder and execute this:
for /R %f in (.\*.pack) do @"%cd%\bin\unpack200" -r -v -l "" "%f" "%~pf%~nf.jar"

Kudos to Nick Russler for figuring out this tricky unpack200 command line!

Determine which Tomcat version is running

Determine process id

First we determine the process id(s) of the running Tomcat instance(s).

We can grep the running process list for ‘catalina.home’:

pgrep -f 'catalina.home'

This might yield more than one pid.

Or we can search by port (8080 is the default, adjust if necessary). The following commands will likely require root privileges:

lsof -t -i :8080

Alternatively, for example if lsof is not installed:

fuser 8080/tcp

Or yet another way, using netstat (or its “ss” replacement):

netstat -nlp | grep 8080

ss -nlp | grep 8080

Determine catalina.home

For the process id(s) determined above, we look at process details:

ps -o pid,uid,cmd -p [pidlist] | cat

For each specified pid, this shows the uid (system user) and the full command line of the process.

Typically the command line will contain something like “-Dcatalina.home=[path]” and that path is the catalina.home system property of the Java process.

Alternatively – with Java 7 and later – we can use the JDK command “jcmd” to query the JVM process for its system properties:

sudo -u [uid] jcmd [pid] VM.system_properties \
   | grep '^catalina.home' \
   | cut -f2 -d'='

Determine version

Now we can finally determine which Tomcat version is installed under the catalina.home path:

[catalina.home]/bin/catalina.sh version \
   | grep '^Server number:'

Note: Please replace [catalina.home] with the path you determined above.

The final output should be something like this:

Server number: 7.0.56.0

Continuous delivery using github, travis-ci and bintray

Continuous-Delivery-schema

Let’s say you work on a Java application and want to frequently make it available for download so that user’s can easily try the latest version.

Let’s say you work primarily on your laptop or personal computer using a Java IDE and commit code changes, but you don’t want to spend time manually building jars, packaging war or zip files, testing your application or uploading files to a website, etc.

Instead you want to have a fully automated process that compiles your source code, runs automated tests and other quality control mechanisms, builds your application and uploads the result to a public website.

But you don’t want to install any infrastructure for this and not run anything besides Java and your IDE on your own machine(s).

Basically you want to use developer-friendly reliable cloud services but you don’t want to pay a single cent.

All of this is possible, as long your code is Open Source:

  • Host your source code on github
  • Let travis-ci run vour build process
  • Let travis-ci upload the build result to bintray

For details, you can take a look at one of my github projects.

Relevant config files:

JVM tips – The G1 Garbage Collector

An old wisdom says that Software can be optimized for latency, throughput or footprint. The same is true for the JVM and its Garbage Collector(s).

Roughly speaking, the classic GC implementations each optimize for one aspect: Serial GC optimizes footprint, Parallel GC optimizes throughput and Concurrent Mark and Sweep (CMS) optimizes for response times and minimal GC induced latency.

But since JDK7u4, we officially have the “Garbage First” (G1) GC. It is still new enough to not even have its own Wikipedia article, but there are good introductory tutorials, articles and tuning guides.

In several ways, G1 is a step up from the conventional GC approaches: It uses non-contiguous heap regions instead of contiguous young and old generations and does most of its reclamation through copying of the live data, thus achieving compaction.

It is based on the principle of collecting the most garbage first and designed with scalability in mind, without compromising throughput.

The benefits of G1 have lead to a proposal and lively debate about Defaulting to G1 Garbage Collector in Java 9.

In conclusion, you can either take the easy path and use the default JVM settings or take some time to learn about modern GC choices and tuning options.

And if you get it all right you might be rewarded with your Java based services performing better than ever before … :)

DBeaver – My new favorite DB tool

I have used Toad for Oracle and Oracle SQL Developer. Those are both good for working with Oracle databases.

However, I generally prefer Open Source tools and ideally something that works with other databases as well.

So I looked around, tried TOra but found it buggy and too limited. Also, its development is quite slow, see commit history.

Then I came across DBeaver and liked it a lot. It is actively developed, the latest version 3.5.1 was actually released 3 days ago.

It is a cross-platform tool (Windows, Linux, MacOS, other Unixes) written in Java, uses the Eclipse framework for a lot of great out-of-the-box features and is overall quite polished.

It supports many databases via JDBC. More details and some comparison with similar tools are mentioned on its About page.

splashscreen-circle