In this blog I will address how to determine if enough osl_server_processes are started; in a future blog discuss how to determine if the max_open_servers value is correct for your environment.
The number of OSL servers is a major factor in the performance of applications that use OSL!
What does it mean to have enough osl_server processes running? It means that when a transaction arrives from another module or system there is always an osl_server that is idle and available to process that transaction. This means that there should never be a time when all the osl_server processes are processing transactions.
The osl_server processes are scheduled in a last in first out manner so the last osl_server process that completes a transaction is the next one scheduled. This means that the earliest time_last_run is the last time that all the servers were busy simultaneously. If the time was in the last day or two, you probably need more servers. If the time is when the system was booted, you probably don’t. Anything in the middle is a judgment call based on how long ago the time was and if you can live with OSL slowdowns every now and then. Figure 1 shows the commands needed to display the time_last_run for several osl_server processes.
as: process -process_name osl_server1; match time_last_run; dump_pte
Using nonrunning process.
Current process is 52, ptep 8D82E500, Overseer.System (osl_server1)
PTE at 8D82E500 for Overseer.System (osl_server1)
time_last_run: 00000000 01834C2E (09-07-20 15:36:51.31266)
as: process -process_name osl_server2; match time_last_run; dump_pte
Using nonrunning process.
Current process is 53, ptep 8D872000, Overseer.System (osl_server2)
PTE at 8D872000 for Overseer.System (osl_server2)
time_last_run: 00000002 462972DC (09-07-22 08:54:17.46376)
as: process -process_name osl_server3; match time_last_run; dump_pte
Using nonrunning process.
Current process is 54, ptep 8D874000, Overseer.System (osl_server3)
PTE at 8D874000 for Overseer.System (osl_server3)
time_last_run: 00000000 0183947E (09-07-20 15:36:51.59513)
. . . .
as: process -process_name osl_server6; match time_last_run; dump_pte
Using nonrunning process.
Current process is 57, ptep 8D87B440, Overseer.System (osl_server6)
PTE at 8D87B440 for Overseer.System (osl_server6)
time_last_run: 00000000 01DBF96A (09-07-20 15:38:19.98936)
as:
|
| Figure 1 – displaying time_last_run |
The macro osl_server_times.cm at the end of this posting creates a file named osl_server_times.(date) holding a sorted list of the time_last_run times. It also displays the file in the terminal window. It assumes that the osl_server processes are named osl_serverN. If not, you can provide your own base name. It does assume that the processes all have a common base name which is what it matches on. The first time listed is the last time that all the servers ran simultaneously. The login time tells you when the process was started; typically they will also be started at the same time which is when the system was booted.
Example 1 shows a module with 6 osl_server processes running. All 6 have had some usage since the system was booted but based on the last run time it probably had something to do with recovery during the boot. Only two processes have been used in the two days since the system was booted. Unless there is an issue with the time is takes the system to boot and be ready to process new transactions I would not add more processes to this module.
osl_server_times.cm run on %phx_vos#m16 at 09-07-22.15:27:10
time_last_run CPU login time process
name number
09-07-20 15:36:51.31266 / 0.87 / 09-07-20 15:33:49 / osl_server1 / 52
09-07-20 15:36:51.31484 / 1.32 / 09-07-20 15:33:50 / osl_server5 / 56
09-07-20 15:36:51.59513 / 0.81 / 09-07-20 15:33:49 / osl_server3 / 54
09-07-20 15:38:19.98936 / 2.51 / 09-07-20 15:33:50 / osl_server6 / 57
09-07-22 08:54:17.46376 / 1.59 / 09-07-20 15:33:49 / osl_server2 / 53
09-07-22 15:26:35.14511 / 2.77 / 09-07-20 15:33:49 / osl_server4 / 55
|
| Example 1 – all processes have some CPU time but no need to add more |
In example 2 the module is running 40 osl_server processes. 13 have not run since the module was booted over 7 months ago so again they have plenty of processes running.
osl_server_times.cm run on %XXXXX#m1 at 11-03-21.15:33:01
time_last_run CPU login time process
name number
10-08-01 07:45:02.68817 / 0.02 / 10-08-01 07:45:02 / osl_server1 / 84
10-08-01 07:45:06.45556 / 0.00 / 10-08-01 07:45:06 / osl_server2 / 85
10-08-01 07:45:10.47752 / 0.00 / 10-08-01 07:45:10 / osl_server3 / 86
10-08-01 07:45:14.51074 / 0.00 / 10-08-01 07:45:14 / osl_server4 / 87
10-08-01 07:45:18.53199 / 0.00 / 10-08-01 07:45:18 / osl_server5 / 89
10-08-01 07:45:22.62286 / 0.01 / 10-08-01 07:45:22 / osl_server6 / 92
10-08-01 07:45:26.67370 / 0.01 / 10-08-01 07:45:26 / osl_server7 / 93
10-08-01 07:45:30.70518 / 0.00 / 10-08-01 07:45:30 / osl_server8 / 94
10-08-01 07:45:38.26467 / 0.02 / 10-08-01 07:45:34 / osl_server9 / 95
10-08-01 07:45:38.77099 / 0.01 / 10-08-01 07:45:38 / osl_server10 / 96
10-08-01 07:45:42.80245 / 0.01 / 10-08-01 07:45:42 / osl_server11 / 97
10-08-01 07:45:46.82534 / 0.00 / 10-08-01 07:45:46 / osl_server12 / 98
10-08-01 07:45:50.85063 / 0.00 / 10-08-01 07:45:50 / osl_server13 / 99
11-03-18 14:33:15.59938 / 9514.24 / 10-08-01 07:46:15 / osl_server19 / 105
11-03-18 14:43:22.17904 / 30208.59 / 10-08-01 07:46:19 / osl_server20 / 106
11-03-18 14:45:00.93608 / 19494.54 / 10-08-01 07:45:58 / osl_server15 / 101
11-03-18 19:29:54.04067 / 37248.83 / 10-08-01 07:46:55 / osl_server25 / 111
11-03-20 00:19:10.75480 / 28961.64 / 10-08-01 07:46:10 / osl_server18 / 104
11-03-20 04:13:02.01803 / 667.23 / 10-08-01 07:45:54 / osl_server14 / 100
11-03-20 07:51:16.66950 / 40151.59 / 10-08-01 07:46:51 / osl_server24 / 110
11-03-20 08:10:21.71397 / 16944.88 / 10-08-01 07:46:02 / osl_server16 / 102
11-03-20 08:12:03.16114 / 39066.37 / 10-08-01 07:46:23 / osl_server21 / 107
11-03-20 08:12:56.92729 / 37474.92 / 10-08-01 07:46:31 / osl_server23 / 109
11-03-20 08:12:56.94239 / 6425.76 / 10-08-01 07:46:06 / osl_server17 / 103
11-03-20 08:12:59.07991 / 33316.21 / 10-08-01 07:46:27 / osl_server22 / 108
11-03-20 08:13:03.05792 / 0.02 / 11-03-20 08:12:59 / osl_server26 / 2317
11-03-20 08:13:16.25518 / 0.02 / 11-03-20 08:13:06 / osl_server28 / 2319
11-03-20 08:13:16.25527 / 0.01 / 11-03-20 08:13:03 / osl_server27 / 2318
11-03-20 08:13:21.60688 / 0.02 / 11-03-20 08:13:16 / osl_server30 / 2321
11-03-20 08:19:09.81250 / 0.15 / 11-03-20 08:13:20 / osl_server31 / 2322
11-03-20 08:50:04.73651 / 0.45 / 11-03-20 08:13:30 / osl_server33 / 2324
11-03-20 21:02:34.17459 / 68.30 / 11-03-20 08:50:49 / osl_server39 / 2406
11-03-21 15:20:20.30186 / 170.07 / 11-03-20 08:50:55 / osl_server40 / 2407
11-03-21 15:32:56.10295 / 211.96 / 11-03-20 08:50:45 / osl_server38 / 2405
11-03-21 15:32:56.10362 / 124.29 / 11-03-20 08:13:10 / osl_server29 / 2320
11-03-21 15:32:59.99746 / 202.52 / 11-03-20 08:13:24 / osl_server32 / 2323
11-03-21 15:33:01.02978 / 175.24 / 11-03-20 08:13:37 / osl_server35 / 2326
11-03-21 15:33:01.35322 / 211.63 / 11-03-20 08:13:33 / osl_server34 / 2325
11-03-21 15:33:01.38417 / 201.76 / 11-03-20 08:50:38 / osl_server36 / 2403
11-03-21 15:33:01.46699 / 194.44 / 11-03-20 08:50:41 / osl_server37 / 2404
|
| Example 2 – some processes have no run since boot, no need to add more |
This final example shows a case where more osl_servers are needed. The module is running 20 servers and all 20 were in use less than 24 hours from the time the macro was run.
osl_server_times.cm run on %XXXXX#m11 at 11-03-21.15:49:03
time_last_run CPU login time process
name number
11-03-20 18:37:18.97851 / 3079.12 / 10-09-10 08:04:38 / osl_server / 56
11-03-20 18:37:18.97999 / 5044.43 / 10-09-10 08:04:37 / osl_server / 48
11-03-20 18:37:22.37568 / 2485.06 / 10-09-10 08:04:37 / osl_server / 51
11-03-20 18:37:26.99446 / 3637.60 / 10-09-10 08:04:37 / osl_server / 52
11-03-20 18:37:26.99464 / 3329.56 / 10-09-10 08:04:38 / osl_server / 62
11-03-20 18:38:04.86495 / 3556.72 / 10-09-10 08:04:37 / osl_server / 54
11-03-20 18:38:48.53218 / 2509.57 / 10-09-10 08:04:38 / osl_server / 63
11-03-20 18:39:06.44256 / 3434.27 / 10-09-10 08:04:39 / osl_server / 67
11-03-20 18:39:06.44291 / 4478.29 / 10-09-10 08:04:38 / osl_server / 64
11-03-20 18:39:06.45211 / 2918.82 / 10-09-10 08:04:37 / osl_server / 49
11-03-20 18:39:06.45213 / 3131.79 / 10-09-10 08:04:37 / osl_server / 53
11-03-21 03:00:35.69830 / 2477.28 / 10-09-10 08:04:38 / osl_server / 58
11-03-21 03:00:35.74118 / 3411.54 / 10-09-10 08:04:38 / osl_server / 59
11-03-21 03:00:36.86657 / 3225.85 / 10-09-10 08:04:38 / osl_server / 61
11-03-21 12:14:28.10003 / 2897.49 / 10-09-10 08:04:38 / osl_server / 60
11-03-21 12:14:28.10185 / 3983.01 / 10-09-10 08:04:38 / osl_server / 57
11-03-21 15:07:43.19355 / 2736.61 / 10-09-10 08:04:37 / osl_server / 50
11-03-21 15:12:11.08222 / 2892.31 / 10-09-10 08:04:38 / osl_server / 66
11-03-21 15:12:13.61254 / 4489.65 / 10-09-10 08:04:38 / osl_server / 65
11-03-21 15:48:52.71038 / 3601.19 / 10-09-10 08:04:38 / osl_server / 55
|
| Example 3 – a case where more servers are needed |
You can increase (or decrease) the number of OSL servers by editing, compiling, and installing two configuration files. These changes will take place at the next reboot. The tuning parameters that control the number of OSL servers are the max_open_servers value in the new_modules.tin and new_backbone_systems.tin. The module_start_up uses the max_open_servers value to calculate the number of osl_server processes to start. If the module is not a bridge module it starts 2 * max_open_servers; if it is a bridge module it will start 4 * max_open_servers.
By setting the ADDITIONAL_OSL_SERVERS value in module_start_up.cm you can start more osl_server processes then the default 2 or 4 times max_open_servers value. You can also just start more processes from the command line without rebooting. The start_more_osl_servers.cm macro, also at the end of this post, makes starting more processes easy to do.
If you have too many osl_server processes running you can stop them but then they will not be there if you ever do need them. In general having extra osl_server processes running does not use a significant amount of resources. However if you want to stop extra processes you should not use the stop_process command. The osl_server process has a switch that tells it to terminate gracefully after it completes the current transaction. The switch, named stop_sw is a short and should be set to -1, see figure 2.
as: process -process_name osl_server1
Using nonrunning process.
Current process is 52, ptep 8D82E500, Overseer.System (osl_server1)
as: set_word stop_sw -1
addr from to
000730F4 0000 FFFF
as:
|
| Figure 2 – stopping an osl_server process |
Because the process terminates only after the current transaction completes it should be set in the process that was last run.
& osl_server_times starts here & & osl_server_times.cm & version 1.0 09-07-10 & version 1.1 09-07-17 check to make sure that the data is on the & expected line of output, if not adjust down 1 line & and correct parsing for process name & version 1.2 09-07-20 added a parameter for matching the osl_server & process name instead of assuming that the name & will be osl_server & version 1.3 10-06-23 added &echo so macro can be used as started & process. Note be sure that the name of the started & process does not match the name of the osl_server & processes or macro will abort. For example, & osl_serverN for the osl_server processes and & osl_server_times for the process running this & macro will cause the macro to abort. & version 1.4 11-03-21 cleaned up some comments prior to publication & & Noah Davids Stratus CAC [email protected] & & &begin_parameters MATCH match_name:string,required,=osl_server &end_parameters & & &echo no_input_lines no_command_lines no_macro_lines & &set_string PD (process_dir) &set_string PL &PD&>process_list &set_string LR &PD&>last_run &set_string LI &PD&>login &set_string CPU &PD&>cpu & &set_string TABLE &PD&>osl_server_times & & & Create a new file or write over an existing file attach_default_output &TABLE& detach_default_output & & &attach_input analyze_system & & & build a file containing a list of all the osl_server processes ..attach_default_output &PL& match &MATCH&; who ..detach_default_output & & & The first line has an "as: " prompt before the output which has to be & stripped. Also the process name is name is in () which gives the & command processor problems since it thinks it is a command function & so translate the ()s to []. Finally translate spaces to underscores so & I don't have to keep using the quote command function. &set LINECOUNT 1 &set_string LINE &+ (ltrim (substr (translate &+ (contents &PL& &LINECOUNT& -hold) '[]' '()') 4)) &set_string LINE (translate (quote &LINE&) '_' ' ') & & & LOOP until we hit a line that is just the as: prompt &while X&LINE&X ^= Xas:X & & & The first space will delimit the process number &set I (calc (index &LINE& '_') - 1) &set PROCESS (substr &LINE& 1 &I&) & & & Look for the [] characters, everything between them is the process name &set I (calc (index &LINE& '[') + 1) &set J (calc (index &LINE& ']') - &I&) &set_string NAME (substr &LINE& &I& &J&) & & & create a temporary file and put the output of the analyze_system & request dump_pte in it. We match on time_last_run so there is only a & few lines in the output file ..attach_default_output &LR&&PROCESS& process &PROCESS& match 'time_last_run' ; dump_pte ..detach_default_output & & ..attach_default_output &LI&&PROCESS& match 'login' ; dump_pte ..detach_default_output & & ..attach_default_output &CPU&&PROCESS& match 'cpu_time:' ; dump_pte ..detach_default_output & & & The time is in line 4. Even though the time is in ()s we don't have & to translate because we are going to extract out the time before the & command processor has a chance to get confused &if (substr (contents &LR&&PROCESS& 4) 2 4) = 'time' &then &set_string LRTIME (substr (contents &LR&&PROCESS& 4) 45 23) &else &do &set_string LRTIME (substr (contents &LR&&PROCESS& 5) 45 23) ..display_line ..display_line process &PROCESS& last run shifted by 1 line &end & & & Now get the login time &if (substr (contents &LI&&PROCESS& 2) 2 5) = 'login' &then &set_string LITIME (substr (contents &LI&&PROCESS& 2) 36 17) &else &do &set_string LITIME (substr (contents &LI&&PROCESS& 3) 36 17) ..display_line ..display_line process &PROCESS& login time shifted by 1 line &end & & & Finally get the CPU time &if (substr (contents &CPU&&PROCESS& 2) 2 3) = 'cpu' &then &set_string CPUTIME (translate &+ (contents &CPU&&PROCESS& 2) '[]' '()') &else &do &set_string CPUTIME (translate &+ (contents &CPU&&PROCESS& 3) '[]' '()') ..display_line ..display_line process &PROCESS& cpu time shiffed by 1 line &end &set_string CPUTIME (translate (quote &CPUTIME&) '_' ' ') &set I (calc (index &CPUTIME& '[') + 1) &set_string CPUTIME (substr &CPUTIME& &I&) &set I (calc (index &CPUTIME& '_') - 1) &set_string CPUTIME (substr &CPUTIME& 1 &I&) & & & Now add a line to our table time - process name - process number ..attach_default_output &TABLE& -append ..display_line &LRTIME& / &CPUTIME& / &LITIME& / &NAME& / &PROCESS& ..detach_default_output & & & increment the line count, read another line from the process list and & translate the ()'s and spaces &set LINECOUNT (calc &LINECOUNT& + 1) &set_string LINE (translate (contents &PL& &LINECOUNT& -hold) '[]' '()') &set_string LINE (translate (quote &LINE&) '_' ' ') &end & & & exit analyze_system quit & & & The table has a bunch of lines that are nothing but as: prompts. The & sort -duplicate_path gets rid of all but the first as: line and also & sorts the rest of the lines so that the process last run longest ago & is first. sort &TABLE& -duplicates_path &PD&>extra & & & use line_edit to remove the as: prompt from the table and also add & column headers &set_string H &+ osl_server_times.cm run on (current_module) at (date).(time) &attach_input line_edit &TABLE& -no_keystrokes -no_backup -no_verbose insert &H& time_last_run CPU login time process name number . change * !as: ! ! * write quit &detach_input & & & copy the table out of the process_dir and into the current dir. &set_string DT (date) copy_file &TABLE& osl_server_times.&DT& -delete display_file osl_server_times.&DT& & & osl_server_times ends here |
| Figure 3 – osl_server_times.cm |
& start_more_osl_servers starts here
&
& start_more_osl_servers.cm
& version 1.0 09-07-21
&
& Noah Davids Stratus CAC [email protected]
&
&
&begin_parameters
BASE base_name:string,required,=osl_server
START starting_index:number,required
END ending_index:number,required
&end_parameters
&
&
& ********** If this is a bridge module, the -super switch should be
& ********** given to each osl_server process.
& ********** Redirect the output of the osl_admin request to a temp file
&
attach_default_output (process_dir)>foo
osl_admin -quit -request_line 'get i_am_bridge'
&set bridge (command_status)
detach_default_output
&
&if &bridge& ^= 0
&then &set OPTS (string -super -syserr)
&else &set OPTS (string -syserr)
&
&
& ********** Use a loop to start the desired number of servers.
&
&set COUNT &START&
&label LOOP
&
&set OUT &BASE&&COUNT&.out
&set CMD (string osl_server &$OPTS& -server_suffix &COUNT&)
&
&
& ********** If it exists, save the old output file.
&
&if (exists &OUT&) ^= 0
&then !rename -delete &OUT& *.old
&
&
& ********** Create a new output file and turn on implicit locking. This
& ********** will allow the file to be displayed while the osl_server is
& ********** still running.
&
!create_file &OUT&
!set_implicit_locking &OUT& on
!start_process &$CMD& -privileged -output_path &OUT& -priority 9 &+
-process_name &BASE&&COUNT&
&
&
& ********** Loop back around until enough osl_server processes have
& ********** been started.
&
&set COUNT &COUNT& + 1
&
&if &COUNT& <= &END&
&then &goto LOOP
&
&
& start_more_osl_servers ends here
|
| Figure 4 – start_more_osl_servers.cm |