Tuesday, September 8, 2020

bash script to rotate logs without restarting process

 The problem with Linux logrotate is that it requires root access to modify its configuration files, and it sends a HUP signal to the process generating the log expecting it to restart and recreate the log file after it has been rotated. I didn't have root access on the server where I needed to rotate logs, and the HUP signal (kill -1) didn't necessarily cause the process to restart and recreate the log file rotated.

The following bash script (saved as rotateLog.sh) can be run as a non-root user and doesn't need the process to restart. It isn't as seamless as logrotate, but it gets the job done when your main priority is to prevent a log file from taking all the drive space.

In crontab, you would invoke it using:

<path to rotateLogs.sh>/rotateLog.sh <path to log file being rotated>

By default it saves up to 20 historical files and rotates if the size of the log file is greater than 20MB, but you can change those numbers as needed:



#!/bin/bash

LOGFILEPATH=$1

MAXLOGS=20
MAXSIZE=20480000c

if [ -z "$1" ]; then
   echo "Not executed. No log file path passed as first parameter."
elif [ ! -f "$LOGFILEPATH" ]; then
   echo "Not executed. Log file path doesn't exist."
else
   PREFIX="${LOGFILEPATH%.*}"
   EXTENSION="${LOGFILEPATH##*.}"
   if [[ $(find $LOGFILEPATH -type f -size +$MAXSIZE 2>/dev/null) ]]; then
      ls -1tdq $PREFIX*.$EXTENSION | tail -n +$MAXLOGS | xargs rm -f
      NOW=$(date +_%y.%m.%d_%H.%M.%S)
      cp $LOGFILEPATH $PREFIX$NOW.$EXTENSION
      cat /dev/null > $LOGFILEPATH
   fi
fi

Tuesday, July 28, 2020

Configure Jenkins for Windows to use https using self signed certificate

1. Open Command Prompt window
2. cd \Program Files (x86)\Jenkins\jre\bin
3. Create self signed certificate: keytool -genkey -keyalg RSA -alias <hostname> -keystore jenkins.jks -validity 3650 -keysize 2048 -dname "CN=<company name>, OU=<organization unit>, O=<company name>, L=<city>, ST=<state>, C=US"
4. Move \Program Files (x86)\Jenkins\jre\bin\jenkins.jks to \Program Files (x86)\Jenkins\secrets\
5. Edit the C:\Program Files (x86)\Jenkins\jenkins.xml file and change the <arguments> block to:

<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=-1 --httpsPort=8443 --httpsKeyStore="%BASE%\secrets\keystore" --httpsKeyStorePassword=your.password.here</arguments>

6. Stop and start the Jenkins service in the Services control panel

Thursday, June 11, 2020

Linux script to email when pem certificate will expire

#!/bin/sh

########################################################
#
#       Check certificates inside a pem formatted file
#
########################################################

# files=($(find /directory -mindepth 1 -maxdepth 1 -name '*.crt'))
declare -a files=("/directory/server.crt" "/directory/server2.crt")

# 30 days in seconds
days=30
limit=$(($days * 24 * 60 * 60))

echo "Checking pem files for certificate(s) expired in less than $days days."

# current time in seconds since epoch
now=$(date "+%s")

# for each file we want to check
for pem in "${files[@]}"; do
   # They expire at this time in seconds since epoch
   enddate=$(openssl x509 -enddate -noout -in "$pem")
   expires_at=$(date -d "$(: | echo $enddate |cut -d= -f 2)" +%s)
   # the difference
   expires_in=$((expires_at - now))
   expires_days=$(($expires_in / 86400))
   # if the certificate will expire in less than limit
   if (( expires_in < limit )); then
      echo "[WARNING] Certificate $pem expires on '$enddate' ($expires_days day(s) remaining)."
      printf "Automated message from server $HOSTNAME:\n[WARNING] Certificate $pem expires on '$enddate' ($expires_days day(s) remaining)." | curl --url 'smtp://smtpserver.com' --mail-from 'me@mail.com' --mail-rcpt 'support_team@mail.com'
   else
      echo "[OK] Certificate $pem expires on '$enddate' ($expires_days day(s) remaining)."
   fi
done

Linux script to email when jks keystore certificate will expire

#!/bin/sh

########################################################
#
#       Check certificates inside a java keystore
#
########################################################
TIMEOUT="timeout -k 10s 5s "
KEYTOOL="$TIMEOUT /opt/atlassian/jira/jre//bin/keytool"
THRESHOLD_IN_DAYS="30"
KEYSTORE=""
PASSWORD=""
RUN=""
RET=0


ARGS=`getopt -o "p:k:t:r" -l "password:,keystore:,threshold:,run:" -n "$0" -- "$@"`

function usage {
        echo "Usage: $0 --keystore keystore [--password password] [--threshold number_of_days_until_expiry] [--run script]"

        exit

}




function start {

        CURRENT=`date +%s`


        THRESHOLD=$(($CURRENT + ($THRESHOLD_IN_DAYS*24*60*60)))

        if [ $THRESHOLD -le $CURRENT ]; then

                echo "[ERROR] Invalid date."

                exit 1

        fi

        echo "Looking for certificates inside the keystore $(basename $KEYSTORE) expiring in $THRESHOLD_IN_DAYS day(s)..."


        $KEYTOOL -list -v -keystore "$KEYSTORE"  $PASSWORD 2>&1 > /dev/null

        if [ $? -gt 0 ]; then echo "Error opening the keystore."; exit 1; fi


        $KEYTOOL -list -v -keystore "$KEYSTORE"  $PASSWORD | grep Alias | awk '{print $3}' | while read ALIAS

        do

                #Iterate through all the certificate alias

                EXPIRACY=`$KEYTOOL -list -v -keystore "$KEYSTORE"  $PASSWORD -alias $ALIAS | grep Valid`

                UNTIL=`$KEYTOOL -list -v -keystore "$KEYSTORE"  $PASSWORD -alias $ALIAS | grep -m 1 Valid | perl -ne 'if(/until: (.*?)\n/) { print "$1\n"; }'`

                UNTIL_SECONDS=`date -d "$UNTIL" +%s`

                REMAINING_DAYS=$(( ($UNTIL_SECONDS -  $(date +%s)) / 60 / 60 / 24 ))

                if [ $THRESHOLD -le $UNTIL_SECONDS ]; then

                        echo "[OK]      Certificate $ALIAS in keystore $KEYSTORE expires on '$UNTIL' ($REMAINING_DAYS day(s) remaining)."

                else

                        echo "[WARNING] Certificate $ALIAS in keystore $KEYSTORE expires on '$UNTIL' ($REMAINING_DAYS day(s) remaining)."

                        printf "Automated message from server $HOSTNAME:\n[WARNING] Certificate $ALIAS in keystore $KEYSTORE expires on '$UNTIL' ($REMAINING_DAYS day(s) remaining)." | curl --url 'smtp://hopkins.tea.state.tx.us' --mail-from 'ITSTOOLSADMIN@tea.texas.gov' --mail-rcpt 'ITSTOOLSADMIN@tea.texas.gov'

                        RET=1

                fi


        done

        echo "Finished..."

        exit $RET

}


eval set -- "$ARGS"


while true

do

        case "$1" in

                -p|--password)

                        if [ -n "$2" ]; then PASSWORD=" -storepass $2"; else echo "Invalid password"; exit 1; fi

                        shift 2;;

                -k|--keystore)

                        if [ ! -f "$2" ]; then echo "Keystore not found: $2"; exit 1; else KEYSTORE=$2; fi

                        shift 2;;

                -t|--threshold)

                        if [ -n "$2" ] && [[ $2 =~ ^[0-9]+$ ]]; then THRESHOLD_IN_DAYS=$2; else echo "Invalid threshold"; exit 1; fi

                        shift 2;;

                -r|--run)

                        if [ ! -f "$2" ]; then echo "Run script not found: $2"; exit 1; else RUN=$2; fi

                        shift 2;;

                --)

                        shift

                        break;;

        esac

done


if [ -f "$RUN" ]; then

        echo "Running script $RUN."

        . $RUN

fi


if [ -n "$KEYSTORE" ]

then

        start

else

        usage

fi

If you name the script above cert-expiry-checker.sh, you would pass a file containing the KEYSTORE and STOREPASS values:

KEYSTORE="/opt/keystore.jks"
PASSWORD=$(echo "cGFzc3dvcmQK"|base64 -di)
PASSWORD="-storepass "$PASSWORD

in a command like /root/src/cert-expiry-checker.sh --run /root/src/set-keystore-password.sh > /root/src/cert-expiry-checker.log

Linux script to email when user password is going to expire

#!/bin/bash
#

for i in  user1 user2 user3
do

  # convert current date to seconds
  currentdate=$(date +%s)

  # find expiration date of user
  userexp=$(chage -l $i | grep 'Password expires' |cut -d: -f2)

  if [[ ! -z $userexp ]]
  then

    # convert expiration date to seconds
    passexp=$(date -d "$userexp" +%s)

    if [[ $passexp != "never" ]]
    then
      # find the remaining days for expiry
      exp=`expr \( $passexp - $currentdate \)`

      # convert remaining days from sec to days
      expday=`expr \( $exp / 86400 \)`

      if [[ $expday -le 30 ]]
      then
                printf "Automated message from server $HOSTNAME:\n[WARNING] Password for Linux user $i will expire on '$userexp' ($expday day(s) remaining)." | curl --url 'smtp://smtp.server.com' --mail-from 'me@email.com' --mail-rcpt 'support_team@email.com'
      fi
    fi
  fi

done

Friday, July 31, 2015

WebSphere Commerce: dbclean causing order items to disappear from production in a staging environment

In a staging environment, WebSphere Commerce has a staging database and a production database. However, the order items are only in the production database because the customers only access the production server(s).

When a catalog entry that is marked for delete is deleted from the staging database, a database trigger inserts an entry in the STAGLOG table which propagates the deletion to the production database the next time stageprop runs. However, since the CATENTRY table is a parent of the ORDERITEMS table, this automatically deletes the order items in the production database linked to the deleted catalog entry. This causes order items to disappear from orders in the production database.

One way to prevent this is to remove the database delete trigger in the staging database on deletion of CATENTRY (and other tables like OFFER) which are parents of the ORDERITEMS table. This causes the running of dbclean on the staging database to delete only the CATENTRY (and other tables like OFFER) and not propagate those deletions to the production database. This means that a separate dbclean would run on the production database to delete those CATENTRY entries marked for delete only if there is no corresponding ORDERITEMS entry.

The flaw in removing the database delete trigger on those tables is that if a CATENTRY (and other tables like OFFER) has dbclean run on it before stageprop has a chance to propagate the change from active to marked for delete status, those entries would remain active in the production database when the corresponding entries in the staging database have been deleted.

Another way to prevent the mysterious disappearance of order items is to retain the database triggers and propagate the user information, orders and order items from the production database back to the staging database before dbclean is run. This ensures that the checks for existing order items linked to CATENTRY (and other tables that are parent to the ORDERITEMS table) done during dbclean occurs on the staging side.

This disappearance of order items caused by dbclean is an obscure problem that is probably not felt by most customers because they tend to have a backend ordering system like Sterling Order Management. Once the order is created in the backend system, there usually is no need to refer back to the order in WebSphere Commerce. The missing order items would stay in WebSphere Commerce and not be propagated to a backend ordering system.

Wednesday, June 3, 2015

Tracing order payment stuck in a state other than PAID in Sterling Order Management

Get the order header key value:

1. Run the following SQL query to get the order header key value if you know the order number:

select ORDER_HEADER_KEY from YFS_ORDER_HEADER where ORDER_NO='WC_999999'

Enable tracing on requestCollection and executeCollection APIs:

1. Open the Application console (https:///smcfs/console/start.jsp)
2. Go to System -> Launch System Management
3. In the System Management Console, go to Tools -> Trace components
4. Click the Add button
5. Choose API for Component Type
6. Choose requestCollection for Component Name
7. Choose VERBOSE for Trace Level
8. Click the Apply button
9. Click the Add button
10. Choose API for Component Type
11. Choose executeCollection for Conponent Name
12. Choose VERBOSE for Trace Level
13. Click on the Apply button

Try to process the order:

1. Open the API tester (https:///smcfs/yfshttpapi/yantrahttpapitester.jsp)
2. In the API Name field, choose requestCollection
3. In the UserId field, enter the same user name that you used to log into the Application or OM console
4. In the Password field, enter the corresponding password
5. In the Message box, enter the following where the value of OrderHeaderKey is the value of ORDER_HEADER_KEY found from the SQL query at the top of this post:



6. Click on Test API Now! button
7. If you get an exception or error in the resulting screen, copy this down
8. If you get no error repeat the above steps except choose executeCollection as the API Name
9. If you still don't get an error or exception after running executeCollection, re-run the above steps for requestCollection again

Turn off the trace by removing the trace entries for requestCollection and executeCollection.
The verbose traces are located in the OM logs directory with a file name of sci.log.Dyyyymmdd.Tnnnnnn where yyyymmdd is the date and nnnnnn is a number that usually starts with 000000 and increments up for each successive log on the same day