3.1.9.2. Metrics Database ontp-tsdb-ml#
3.1.9.2.1. Enable Default Machine Learning Code / Testing#
3.1.9.2.1.1. Machine Learning Notes:#
Note
By default the Machine Learning code is not enabled to run automatically.
In order to execute the built in ontp ml example code you will have to enable a pg_cron job to call the relevant functions.
Warning
Please make sure that you have enough memory and postgresql worker processes to execute any machine learning code.
3.1.9.2.1.2. ML Functions#
Example ML Code distributed with ontp-tsdb-ml
3.1.9.2.1.3. ARIMA payload forecast example#
- Function: tcp_payload_forecast_5min()
cap_limit integer DEFAULT 1000
miter integer DEFAULT 10
tbucket INTERVAL DEFAULT ‘5 seconds’
cinterval INTERVAL DEFAULT ‘60 minutes’
Execute via pg_cron
select cron.schedule('*/5 * * * *',
$$CALL tcp_payload_forecast_5min(100000,10,'5 seconds','60 minutes');
CALL tcp_payload_forecast_5min_copy();$$);
3.1.9.2.1.4. Test Tensorflow#
- Function: test_tf()
cap_limit integer DEFAULT 1000
miter integer DEFAULT 10
tbucket INTERVAL DEFAULT ‘5 seconds’
cinterval INTERVAL DEFAULT ‘60 minutes’
Execute via postgresql connection
CALL test_tf(1000,10,'5 seconds','30 minutes');
3.1.9.2.1.5. Decision Tree For TCP Flag Data#
- Function: tcp_flags_decision_tree5min()
cap_limit integer DEFAULT 1000
tbucket INTERVAL DEFAULT ‘5 seconds’
cinterval INTERVAL DEFAULT ‘60 minutes’
Execute via pg_cron
select cron.schedule('*/5 * * * *',
$$CALL tcp_flags_decision_tree5min(100000,'5 seconds','60 minutes');
SELECT tcp_flags_decision_tree5min_copy();$$);
3.1.9.2.1.6. Random Forrest For TCP Flag Data 1#
- Function: tcp_flags_random_forest5min()
cap_limit integer DEFAULT 1000
tbucket INTERVAL DEFAULT ‘5 seconds’
cinterval INTERVAL DEFAULT ‘60 minutes’
Training Method
SELECT madlib.forest_train(
'metrics_ref_tcp_rforest_train', -- source table
'ref_tcp_rforest_train_output', -- output model table
'convo_id', -- id column
'class', -- response
'ip_flags,ip_len,ip_hlen,tcp_plen,tcp_hlen,tcp_flags,tcp_flags_ns,tcp_flags_ack,tcp_flags_cwr,tcp_flags_ecn,tcp_flags_fin,tcp_flags_res,tcp_flags_syn,tcp_flags_urg,tcp_flags_push,tcp_flags_reset,tcp_exp_group,tcp_exp_severity', -- features
NULL, -- exclude columns
NULL, -- grouping columns
20::integer, -- number of trees
2::integer, -- number of random features
TRUE::boolean, -- variable importance
1::integer, -- num_permutations
8::integer, -- max depth
3::integer, -- min split
1::integer, -- min bucket
10::integer -- number of splits per continuous variable
);
Execute via pg_cron
select cron.schedule('*/5 * * * *',
$$CALL tcp_flags_random_forest5min(100000,'5 seconds','60 minutes');
SELECT tcp_flags_random_forest5min_copy();$$);
3.1.9.2.1.7. Random Forrest For TCP Flag Data 2#
- Function: tcp_flags_random_forest5min()
cap_limit integer DEFAULT 1000
tbucket INTERVAL DEFAULT ‘5 seconds’
cinterval INTERVAL DEFAULT ‘60 minutes’
Training Method
SELECT madlib.forest_train(
'metrics_ref_tcp_rforest_train', -- source table
'ref_tcp_rforest_train_output_1a', -- output model table
'convo_id', -- id column
'class', -- response
'tcp_plen,tcp_hlen,tcp_flags,tcp_flags_ns,tcp_flags_ack,tcp_flags_cwr,tcp_flags_ecn,tcp_flags_fin,tcp_flags_res,tcp_flags_syn,tcp_flags_urg,tcp_flags_push,tcp_flags_reset,tcp_exp_group,tcp_exp_severity', -- features
NULL, -- exclude columns
NULL, -- grouping columns
20::integer, -- number of trees
2::integer, -- number of random features
TRUE::boolean, -- variable importance
1::integer, -- num_permutations
8::integer, -- max depth
3::integer, -- min split
1::integer, -- min bucket
10::integer -- number of splits per continuous variable
);
Execute via pg_cron
select cron.schedule('*/5 * * * *',
$$CALL tcp_flags_random_forest5min(100000,'5 seconds','60 minutes');
SELECT tcp_flags_random_forest5min_copy();$$);