Getting multiple JBoss 4.x instances to run in a multihomed server

If you have configured your server to be multihomed (server multiple IP addresses), to get multiple JBoss instances to serve each IP address per JBoss instance without traffic conflict, you need to do the following:

1. Modify run.sh scripts to associate one JBoss instance to one IP address and another JBoss instance to another IP address (add -b right before "$@")

2. If using one of the jboss_init_.sh scripts for autostart on boot, modify the "# Provides: jboss" line so that each instance has a unique name. Multiple scripts can't have the same provider name on the same box. Use the version of the script in 4.2.3 which doesn't require passing the username and password in the command line to stop JBoss when JBoss is secured. To make these scripts available to users so that those who can sudo su - jboss can start and stop the servers:

a. cp /etc/init.d/[script name] /sbin
b. Modify script in sbin to sudo su - jboss instead of su - jboss
c. Delete the line, "cd $JBOSS_HOME/bin"

3. For each JBoss instance, add or edit the [JBOSS_HOME]/server/default/conf/jndi.properties file and make sure it contains the following three lines:

java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
java.naming.provider.url=[IP address]:1099

4. Create a [JBOSS_HOME]/server/default/conf/multihomedprops directory and copy the above jndi.properties file into it

5. Edit the [JBOSS_HOME]/bin/run.sh file and add the following line after the last section of code where the $JBOSS_CLASSPATH variable is modified in any way:

JBOSS_CLASSPATH="$JBOSS_CLASSPATH:$JBOSS_HOME/server/default/conf/multihomedprops"

Running web server as non-root in Linux and still use ports 80 and 443

1. As root, run the following commands assuming web server is listening on ports 8080 and 8443:

iptables -t nat -A PREROUTING -p tcp -d [IP address] --dport 80 -j DNAT --to-destination [IP address]:8080
iptables -t nat -A PREROUTING -p tcp -d [IP address] --dport 443 -j DNAT --to-destination [IP address]:8443

I like this format of the iptables command because if the server is multihomed to server multiple IP addresses, you can use these same commands over for each IP address and make sure traffic from one IP address won't seep to the other.

2. Add these commands to the /etc/init.d/boot.local file to make sure they are executed on boot

NOTE: iptables is primarily a firewall tool, so if an application is running on the same box as the web server, trying to call the box by its hostname or IP address will fail because iptabled thinks the source being localhost and the destination not being local is a spoof attempt. localhost to localhost invocation is allowed.

How to decode CMS keyfile password stored in stash file (.sth)

1. Get unstash.jar

2. Set JAVA_HOME=[location of WebSphere java directory]

3. Assuming PATH points to the java bin diretory, enter java -jar unstash.jar filename.sth

Website to decode WebSphere encoded passwords

http://www.sysman.nl/wasdecoder/

Eliminating WebSphere Application Server 6.x & 7.x web server plugin with another load balancer

There are situations where you would forego installing a web server in front of the application server(s) for WebSphere Application Server 6.x & 7.x. To make it work, the following three steps need to be taken:

1. Add the following web container custom properties for each application server:

trusthostheaderport = true
com.ibm.ws.webcontainer.extractHostHeaderPort = true

This makes sure that the HTTPResponseHeader.sendRedirect() API call will send back a URL using the port in the host header of the request message rather than the HTTP/S port that the application server listens to. The default behavior of the application server doesn't match the default behavior of the web server plugin who AppServerPortPreference setting in the plugin-cfg.xml file is hostHeader by default.

2. (Optional) Add the RemoveServerHeader = true custom property to each application server's inbound HTTP transport channel for both the secure (HTTPS) and unsecure transport chains. By default the transport chains are named WCInboundDefault for HTTP and WCInboundDefaultSecure for HTTPS. This removes the response message header saying that the server is a WebSphere Application Server. Since the web server is no longer masking this with its own server header, it is best to eliminate this for security purposes.

3. Assuming the web server being replaced is the IBM HTTP Server and it was serving HTTPS traffic, you need to convert the CMS keyfile (filename suffix of kdb) used by the web server to JKS and set the application server's HTTPS port to use the new JKS file:

a. Run "gsk7cmd(.sh) -keydb -convert -db key.kdb -pw [password]
-old_format cms -new_format JKS" assuming the keyfile name is key.kdb

b. Review the converted JKS file and eliminate any expired certificates or certificates that weren't the default certificate being served in the original kdb file because unlike CMS keyfiles, JKS keyfiles don't have a way to flag which certificate is the default certificate.

c. Copy over the JKS file to a common relative location on the deployment manager and application server boxes (e.g., /opt/WebSphere/AppServer/profiles/Dmgr01/etc and /opt/WebSphere/AppServer/profiles/AppSrv01/etc)

d. In 6.x, create a new SSL repertoire, set the JKS file as both the key and trust files, and point it to the relative location of the JKS file (e.g., ${USER_INSTALL_ROOT}/etc/key.jks). In the application server's WCInboundDefaultSecure transport chain, change the SSL transport channel to use the newly created SSL repertoire.

e. In 7.x, create a new keystore and point it to the relative location of the JKS file. Create a new SSL configuration and assign the newly created keystore as both the trust store and keystore. In the application server's WCInboundDefaultSecure transport chain, change the SSL transport channel to use the newly created SSL configuration.

f. Make sure the nodes are synced and restart the application server(s).

4. In the new load balancer which will replace the web server plugin, make sure to enable sticky sessions (a.k.a. session affinity).

Decode password in WebSphere Application Server 6.x

cd <WAS home>/lib
../java/bin/java -cp securityimpl.jar:iwsorb.jar:ras.jar:wsexception.jar:bootstrap.jar:emf.jar:ffdc.jar com.ibm.ws.security.util.PasswordDecoder {xor}<encoded password>