Technology – No Place for Wimps – My daily dealings with programming and technology

Jason Hendriks
2021-10-04

Reading Spring Profile-aware properties from shared Jars

In projects I’ve been on, it’s very common to have a Common or Shared project/module where shared code is kept. For example, you might have a IngestData project and a DigestData project and they both require a Common project where some common business logic lives. In this post, I’ll show how to use Spring to load properties not only from your application project, but your common project as well, and honour Spring profiles at the same time.

First, why does this not work out of the box?

Spring automatically picks up your application.properties or application.yml file from your main application, and will even parse that file for Profiles, or load other Profile-specific properties files that share the same root (such as application-UAT.properties). But if you put an application.properties file in your common project, it will be ignored.

Your first thought might be to use @PropertySource or @PropertySources, but this annotation loads an exact file name only, and will not load your property files with Profiles in the names. There is an open issue to make this a feature request at https://github.com/spring-projects/spring-boot/issues/24688, but the Spring team has some understable concerns about this. @PropertySource is discouraged for this use anyway. The official documents says:

While using @PropertySource on your @SpringBootApplication may seem to be a convenient way to load a custom resource in the Environment, we do not recommend it. Such property sources are not added to the Environment until the application context is being refreshed. This is too late to configure certain properties such as logging.* and spring.main.*which are read before refresh begins.
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.application.customize-the-environment-or-application-context

But the docs do mention how to accomplish this with a custom EnvironmentPostProcessor, and combining this code with a custom YAML PropertySourceFactory (https://www.baeldung.com/spring-yaml-propertysource) we can accomplish our desired behaviour:

public MyEnvironmentPostProcessor implements EnvironmentPostProcessor {

  @Override
  public void postProcessEnvironment(final ConfigurableEnvironment, final SpringApplication application) {
      environment.getPropertySources().addLast(createYamlResource("common.yml");
      environment.getPropertySources().addLast(createPropertiesResource("common.properties");
environment.getPropertySources().addLast(createPropertiesResource("common-UAT.properties");
    }

  private Resource createYamlResource(String name) {
      Resource path = new ClassPathResource(name);
      EncodedResource encodedResource = new EncodedResource(path);
      YamlPropertySourceFactory factory = new YamlPropertySourceFactory();
      return factory.createPropertySource("name", encodedResource);
  }

  private Resource createPropertiesResource(String name) {
      return new ClassPathResource(name);
  }

  private Class YamlPropertySourceFactory {

    @Override
    public PropertySource<?> createPropertySource(@Nullable final String name, final EncodedResource resource) throws IOException {

      YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
      factory.setResources(resource.getResource());
      factory.afterPropertiesSet();
      Properties properties = factory.getObject();
      String sourceName = name != null ? name : resource.getResource().getFilename();
      return new PropertiesPropertySource(sourceName, properties);
  }

}

Make sure the properties file exist: common.yml, common.properties, common-UAT.properties because the code above will not fail gracefully.

No Comments InUncategorized

Jason Hendriks
2021-02-26

VoIP with Edgerouter ER-X

I finally decided to give up my Apple AirPort Extreme base station. This device has performed fabulously for over ten years. However it is starting to drop from the LAN occasionally.

Over the holidays I purchased an Ubiquiti Edgerouter ER-X to take over routing duties from the Airport Extreme. However I can no longer receive VoIP calls!

I have a typical home/SMB LAN setup. NAT allows my private internal devices to communicate with the Internet, so I shouldn’t need to have any custom firewall rules or port forwarding. Turned out my problem was SIP ALG.

An ALG, or Application Layer Gateway, helps traffic move across a NAT. But most SIP traffic doesn’t need help, and ironically, SIP ALG can end up corrupting SIP traffic routing instead of helping it route properly.

My AirPort Extreme didn’t support SIP ALG, because that’s a fairly advanced NAT function for some very specific setups (like STUN with symmetric NAT).

A quick look at my Edgerouter’s configuration confirmed that SIP ALG is enabled:

Linux ubnt 4.14.54-UBNT #1 SMP Wed Oct 28 16:53:18 UTC 2020 mips
Welcome to EdgeOS
ubnt@ubnt:~$ lsmod | grep sip
nf_nat_sip              7152  0
nf_conntrack_sip       17597  1 nf_nat_sip
nf_nat                 14044  8 nf_nat_pptp,nf_nat_proto_gre,nf_nat_h323,nf_nat_sip,nf_nat_ftp,nf_nat_masquerade_ipv4,nf_nat_ipv4,nf_nat_tftp
nf_conntrack           62887  18 nf_nat_pptp,nf_conntrack_sip,nf_nat_h323,nf_conntrack_ftp,nf_nat_sip,nf_conntrack_ipv4,nf_conntrack_tftp,ipt_MASQUERADE,nf_conntrack_pptp,nf_nat_ftp,nf_conntrack_proto_gre,xt_CT,nf_nat_masquerade_ipv4,nf_conntrack_h323,xt_conntrack,nf_nat_ipv4,nf_nat_tftp,nf_nat

That’s easy enough to disable

ubnt@ubnt:~$ configure
[edit]
ubnt@ubnt# set system conntrack modules sip disable
[edit]
ubnt@ubnt# commit
[edit]
ubnt@ubnt# save
Saving configuration to '/config/config.boot'...
Done
[edit]
ubnt@ubnt# exit
exit
ubnt@ubnt:~$ lsmod | grep sip
ubnt@ubnt:~$

Hey!! 🙂 The phone’s ringing.

If you want some quiet-time again 😉 just re-enable:

ubnt@ubnt:~$ configure
[edit]
ubnt@ubnt# set system conntrack modules sip enable-indirect-media
[edit]
ubnt@ubnt# set system conntrack modules sip enable-indirect-signalling
[edit]
ubnt@ubnt# commit
[edit]
ubnt@ubnt# save
Saving configuration to '/config/config.boot'...
Done
[edit]
ubnt@ubnt# exit

No Comments InUncategorized

Jason Hendriks
2021-02-22

Creating a Custom Login Form with Spring Boot

I recently had to add a pure HTML login form to an existing Spring Boot application, integrating with Spring Security. All the top posts in Google mentioned how to do it with Thymeleaf, but nothing with just pure HTML. It took me about three hours of reading documentation and trail and error to get it.

I followed the official Spring IO Guide to get started. I created a simple Spring Boot app at Spring Initializr configured with Spring Web, Thymeleaf and Spring Security. My pom.xml ended up looking like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.4.3</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>demo</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>demo</name>
	<description>Demo project for Spring Boot</description>
	<properties>
		<java.version>11</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-security</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-thymeleaf</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.thymeleaf.extras</groupId>
			<artifactId>thymeleaf-extras-springsecurity5</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.security</groupId>
			<artifactId>spring-security-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

The first thing I did was edit the @SpringBootApplication class file to disable the provided security settings so I could customize my own.

package ca.hendriks.logindemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;

@SpringBootApplication(exclude = { SecurityAutoConfiguration.class })
public class LoginApplication {

	public static void main(String[] args) {
		SpringApplication.run(LoginApplication.class, args);
	}

}

Configure app for a static HTML login form

The next step was to create the Spring Security configuration. This is where the login form is specified.

package ca.hendriks.logindemo;

import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@EnableWebSecurity
public class LoginConfiguration extends WebSecurityConfigurerAdapter {

    @Override  
    public void configure(HttpSecurity http) throws Exception {  
    	configureLogin(http);
    }  

    private void configureLogin(HttpSecurity http) throws Exception {
        http  
        .csrf().disable()
        .authorizeRequests()  
        .anyRequest().authenticated()  
        .and()
        .formLogin()
        	.loginPage("/html_login.html")
        	.loginProcessingUrl("/doLogin")
        	.failureUrl("/login-error.html")
        	.defaultSuccessUrl("/welcome", true)
        	.permitAll();
	}

    @Override  
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {  
        auth.inMemoryAuthentication()  
            .withUser("user")  
            .password("{noop}pass")  
            .roles("USER");  
    }  
    
}

The overridden configure method just sets up a username and password to login with. Obviously in a real system, you would do something actually useful.

There’s a lot going on in there, so let’s break it down.

.csrf().disable()

From what I can tell, Spring’s Cross-Site Request Forgery solution is to add a nonce to the login page. Smart. But I can’t use it with a pure, static HTML solution, so I’ve disabled it.

.loginPage("/html_login.html")

When Spring decides it’s time to ask for a user’s name and password, it forwards to this URL. This is where our pure, static HTML file with the simple login form will go. Default is “/login”.

.loginProcessingUrl("/doLogin")

This is the URL Spring will expect the username + password parameters to be found (either GET or POST). Default is “/login”, but I found debugging easier using a unique URL.

.failureUrl("/login-error.html")

By default, Spring will redirect to the loginPage and append "?error" onto the URL. This means you’ll need to add Javascript to detect this and display an error message to the user. Instead, I opted to just go to a different page with a hard-coded error message. No Javascript required.

.defaultSuccessUrl("/welcome", true)

This is obviously where Spring forwards to after a successful login. What’s isn’t obvious is the boolean parameter. true tells Spring to forget where the user was before and force a forward to the specified page.

That’s it! Finally we can add our two HTML pages into the “src/main/resources/static” directory. And test our work.

/src/main/resources/static/html_login.html

<html>
<head></head>
<body>
    <div>
        <form name="f" action="/doLogin" method="post">
            <fieldset>
                <legend>Please Login</legend>
                <label for="username">Username</label>
                <input type="text" id="username" name="username"/>        
                <label for="password">Password</label>
                <input type="password" id="password" name="password"/>    
                <div class="form-actions">
                    <button type="submit" class="btn">Log in</button>
                </div>
            </fieldset>
        </form>
    </div>
  </body>
</html>

/src/main/resources/static/login-error.html

<html>
    <body>
        <p>
            DENIED<br/>
            <a href="/">Try again</a>
        </p>
    </body>
</html>

Running the Spring Boot application and pointing my web browser to http://localhost:8080/ :

Great success! But I don’t really like the hardcoded error message, nor disabling CSRF. I decided to try the Thymeleaf version and let management see the benefit for themselves.

Configure app for a dynamic HTML (Thymeleaf template) login form

First, we’ll need to tweak our custom LoginConfiguration class:

    private void configureLogin(HttpSecurity http) throws Exception {
        http  
        .authorizeRequests()  
        .anyRequest().authenticated()  
        .and()
        .formLogin()
        	.loginPage("/login")
        	.loginProcessingUrl("/doLogin")
        	.defaultSuccessUrl("/welcome", true)
        	.permitAll();
	}

The loginPage is set to the default “/login”. But we’re going to intercept this call using a Spring Web @Controller to inject error messages and whatnot. We’ll need to add another class:

package ca.hendriks.logindemo;

import java.time.LocalTime;
import java.time.format.DateTimeFormatter;

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
public class LoginController {

    @RequestMapping("/welcome")  
    @ResponseBody  
    public String index() {  
    	String time = LocalTime.now().format(DateTimeFormatter.ISO_LOCAL_TIME);
        return "Hello, world! Time is " + time;  
    }  

	@RequestMapping(value = "/login", method = RequestMethod.GET)
    public String thymeLeafLogin(Model model, String error, String logout) {
        if (error != null)
            model.addAttribute("errorMsg", "Your username and password are invalid.");

        if (logout != null)
            model.addAttribute("msg", "You have been logged out successfully.");

        return "thyme_login.html";
    }
	
}

The “/login” @RequestMapping we just wrote will inspect the Model for error messages and pass them on to the Thymeleaf template for display. The method ends by returning our destination – the Thymeleaf template file, which must be created in the templates directory of the class path.

/src/main/resources/templates/thyme_login.html

<html xmlns:th="https://www.thymeleaf.org">
<head></head>
<body>
    <div th:fragment="content">
        <form name="f" th:action="@{/doLogin}" method="post">               
            <fieldset>
                <legend>Please Login</legend>
                <div th:if="${param.error}" class="alert alert-error">    
                    Invalid username and password.
                </div>
                <div th:if="${param.logout}" class="alert alert-success"> 
                    You have been logged out.
                </div>
                <label for="username">Username</label>
                <input type="text" id="username" name="username"/>        
                <label for="password">Password</label>
                <input type="password" id="password" name="password"/>    
                <div class="form-actions">
                    <button type="submit" class="btn">Log in</button>
                </div>
            </fieldset>
        </form>
    </div>
  </body>
</html>

Running the Spring Boot application and pointing my web browser to http://localhost:8080/ :

Dynamic HTML (Thymeleaf template) login form

Dynamic HTML (Thymeleaf template) login error

As the Spring article mentions, CSRF is also needs to logout, so you will have to use another Thymleaf template with a form to submit the logout request.

I feel that Thymeleaf is definitely the better way to go, but make sure you create the template file in the {CLASSPATH}/templates directory.

No Comments InUncategorized

Jason Hendriks
2021-01-28

Creating SSH keys for Ubiquiti Edgerouter (in macOS)

SSH is an application that allows you to remotely login and access another computer’s terminal. Like telnet, but with encryption. Usually the SSH application on the remote side will demand a login and password before letting you in.

There’s a quicker, more secure way of logging in if you take the time to set it up. By creating SSH keys (shared by both sides of the connection), SSH will login without requiring your password. Here’s how to set that up (in macOS Big Sur).

1. Generate the SSH keys

The SSH keys are a public/private pair (aka asymmetric cryptography). The private key should never be shared, and this one will live on the computer you are connecting from. The public key should be moved or copied to the computer you are connecting to.

In Terminal, enter the following command:

ssh-keygen -t rsa

When prompted, accept the default for file location and type in a password (optional – I never do – but recommended). ssh-keygen will generate two files for you:

Your private key will be at .ssh/id_ra
Your public key will be at .ssh/id_rsa.pub

2. Copy your public key to the Edgerouter.

I’m going to temporarily dump mine in the /var/tmp directory. Substitute your own user and host for “ubnt” and “ubnt.local” below:

scp ~/.ssh/id_rsa.pub ubnt@ubnt.local:/var/tmp/

Close the Terminal – none of the remaining commands below are entered into the Mac!

3. Import the file into the Edgerouter configuration

SSH into the Edgerouter, and substitute your own user and filename for “ubnt” and “id_rsa.pub” below:

configure
load key ubnt /var/tmp/id_rsa.pub
set service ssh disable-password-authentication
commit
save

4. You’re Finished!

The public key will be used to encrypt messages that can only be decrypted with the private key. When you SSH to the target machine, it will create a secret message and when the requesting machine proves that it can read the message – voila! – you are allowed to login sans password.

5. Technology – No Place for the Weak

Now of course trouble started when I tried to upload an updated public key. Each time I ran the “load key” command, the Ubiquiti responded with

Cannot open configuration file /config/key: No such file or directory

I tried deleting the key – didn’t work:

delete system login user ubnt autentication public-keys
commit
save

I tried re-enabling password authentication – didn’t help:

delete service ssh disable-password-authentication
commit
save

Even trying to load the key via the GUI didn’t work. Whatever this /config/key file is, it is important! Where the hell did it go??

I finally found more robust commands that allowed me to import a new key:

set system login user ubnt authentication public-keys ubnt key AAAAB3NzaC1yc2EAAAADAQABAAABgQDIjd4bDv35Yk0M8zePhKkuDIfcpBjD0PGb93L+gV2SIpWNHmCRffRQjZ5NJYLs95eMuCLY/761shjf+mmxnO7PZvFSWnsxgqO4sTZgbM/NUEquQy88peHreg3xli3IdBhImNCdFpNmgACqEiRPVuJ0Q6zeiym0zqKw8I7QGD0w2qjstzn6YCyDuzw04rHLW8eHffZAXmOp5AzSwc1VmegxWkh8Yp/Nptt6hZ9of68kCfJCX3Eiad/GFKVrvXULypkllSlNbBAfLJ+k/3NuJNxDruODyBSDMSlbGKB0H5uEhrAxLkzDSRvrNSgOboyi+D86708kvMlKKnRwQj8xg4aHsYz4a5TSWSne4dK3ttkbBFXv4PNdGCgVutm0PjACiq2Ck51c24o7rdzGbgXiDn/mF8k292KJD4ia4cNK3zZOxSy+o0iBvsmncBLHsHqQopDTMgxJYkOAckYe2kHKy3YLGONF82utTp5f70AHNSMq2ttw6hGzDrr1tsUaM/VOPZ8=

set system login user ubnt authentication public-keys ubnt type ssh-rsa

YMMV.

No Comments InUncategorized

Jason Hendriks
2021-01-25

Hello world!

Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

class HelloWorld {
}

One comment InUncategorized