Using Python to cleanup Planning Analytics(a.k.a., IBM TM1) log file directory

One of the small inconveniences IBM Planning Analytics(a.k.a., IBM TM1) administrators face is cleaning up of the Log file folder.

Planning Analytics application tends to create new log files with different suffixes once the log file size hits the Max File setting set in the TM1s configuration file.

For example, If Audit logging is enabled then, Once the audit file, which the system is currently writing into reaches Max File setting, then the application creates a new file with a different suffix( as displayed below) and start recording into that. 

• tm1auditstorexxxxxxxxxxxx size: 9472 kb <— current file system is writing into.
• tm1auditstorexxxxxxxxxxxx size: 200000 kb <— prior audit file.
• tm1auditstorexxxxxxxxxxxx size: 200000 kb <— prior audit file.

As you can see it can get quite congested within a short time frame in any decent Production Environment. As an admin, you have to make a practice of going into the server and manually moving/deleting them.

Here is where Python comes in handy!!

The program listed below does the following.

• Looks for all files within a folder that matches the listed pattern (i.e., tm1auditstore*.log) and makes a list.
• Identifies the latest log file from that list.
• Loops thru the list, and moves all the files except for the latest log file to Destination folder mentioned in the program.

This python program can be scheduled via scheduler of your choice. I used this code for cleaning up Planning Analytics log files, but this can be used to move any file for one place to another with a slight tweak to the code.

'''
 The below program checks the listed source folder for files matching the pattern listed below.
 In this case"tm1auditstore*.log", except for the latest file matching the description, all others will be moved 
 into the destination directory.
 The reason we do not move the latest file is cause, in case of TM1/Planning Analytics system is actively writing
 into the latest file.
'''
import glob
import os
import shutil

#location of the log files
logFilePath = 'C:\Projects\Python\TM1Log\Testing'

#location we want to move the log files to.
targetFolderPath =  'C:\Projects\Python\TM1Log\Testing\Moved'

fileCount = 0
#below identifies the log file with minimum timestamp and moves it into different folder. so we dont clog the main file directory.
list_of_files = glob.glob(os.path.join(logFilePath, "tm1auditstore*.log")) # * means all if need specific format then *.csv
latest_file = max(list_of_files, key=os.path.getctime)
#print( latest_file)  #for testing purpose.

#looping thru the list of files

for items in list_of_files:
       #print( items )- for validation
       if items != latest_file: #making sure we dont move the latest file system is writing into.
           shutil.move( items ,targetFolderPath)
           fileCount= fileCount + 1 
#couting the files moved for validation purpose
    
#print(fileCount)  # for validation