[2020/09/17-15:54:02.802] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction]  - ******************************************
[2020/09/17-15:54:02.803] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction]  - ** Start train session s1
[2020/09/17-15:54:02.803] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction]  - ******************************************
[2020/09/17-15:54:02.813] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.data] T-MbtGxk2T - [ct: 23] Need to compute sampleId before checking memory cache
[2020/09/17-15:54:02.813] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dip.shaker.runner] T-MbtGxk2T - [ct: 23] Script settings sampleMax=104857600 processedMax=-1
[2020/09/17-15:54:02.813] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dip.shaker.runner] T-MbtGxk2T - [ct: 23] Processing with sampleMax=104857600 processedMax=524288000
[2020/09/17-15:54:02.814] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dip.shaker.runner] T-MbtGxk2T - [ct: 24] Computed required sample id : 548dae92f45047aa9dbe566126cd918c-NA-b9689059e8435a4efb7ef594c85e26650--d751713988987e9331980363e24189ce
[2020/09/17-15:54:02.814] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.shaker.cache] T-MbtGxk2T - Shaker MemoryCache get on dataset CREDITCARDFRAUDSTART3.transactions_known key=ds=9fd59102cf72edf317373c7eba183593--scr=1ef6339241330ea8d518ac6295a7f69e--samp=548dae92f45047aa9dbe566126cd918c-NA-b9689059e8435a4efb7ef594c85e26650--d751713988987e9331980363e24189ce: hit
[2020/09/17-15:54:02.814] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 24] Column transaction_id meaning=LongMeaning fail=0
[2020/09/17-15:54:02.815] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 25] Column authorized_flag meaning=LongMeaning fail=0
[2020/09/17-15:54:02.815] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 25] Column purchase_date meaning=DateSource fail=0
[2020/09/17-15:54:02.815] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 25] Column purchase_date_parsed meaning=Date fail=0
[2020/09/17-15:54:02.815] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 25] Column purchase_year meaning=LongMeaning fail=0
[2020/09/17-15:54:02.815] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 25] Column purchase_month meaning=LongMeaning fail=0
[2020/09/17-15:54:02.816] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 26] Column purchase_day meaning=LongMeaning fail=0
[2020/09/17-15:54:02.816] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 26] Column purchase_dow meaning=LongMeaning fail=0
[2020/09/17-15:54:02.816] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 26] Column purchase_weekend meaning=LongMeaning fail=0
[2020/09/17-15:54:02.816] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 26] Column purchase_hour meaning=LongMeaning fail=0
[2020/09/17-15:54:02.816] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 26] Column card_id meaning=Text fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column merchant_id meaning=Text fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column merchant_category_id meaning=LongMeaning fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column item_category meaning=Text fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column purchase_amount meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column signature_provided meaning=LongMeaning fail=0
[2020/09/17-15:54:02.817] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 27] Column card_first_active_month meaning=DateSource fail=0
[2020/09/17-15:54:02.818] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 28] Column card_first_active_month_parsed meaning=Date fail=0
[2020/09/17-15:54:02.818] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 28] Column days_active meaning=LongMeaning fail=0
[2020/09/17-15:54:02.818] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 28] Column card_reward_program meaning=Text fail=0
[2020/09/17-15:54:02.818] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 28] Column card_latitude meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.818] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 28] Column card_longitude meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.819] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 29] Column card_fico_score meaning=LongMeaning fail=0
[2020/09/17-15:54:02.819] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 29] Column card_age meaning=LongMeaning fail=0
[2020/09/17-15:54:02.819] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 29] Column merchant_subsector_description meaning=Text fail=0
[2020/09/17-15:54:02.819] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 29] Column merchant_latitude meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.819] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 29] Column merchant_longitude meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.820] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 30] Column merchant_location meaning=GeoPoint fail=0
[2020/09/17-15:54:02.820] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 30] Column merchant_cardholder_distance meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.820] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 30] Column merchant_state meaning=USStateMeaning fail=0
[2020/09/17-15:54:02.820] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 30] Column merchant_state_enName meaning=USStateMeaning fail=0
[2020/09/17-15:54:02.820] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 30] Column card_location meaning=GeoPoint fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column card_state meaning=USStateMeaning fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column card_state_enName meaning=USStateMeaning fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column purchase_amount_min meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column purchase_amount_max meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column purchase_amount_avg meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.821] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 31] Column w2_purchase_amount_min meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.822] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 32] Column w2_purchase_amount_max meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.822] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.shaker.schema] T-MbtGxk2T - [ct: 32] Column w2_purchase_amount_avg meaning=DoubleMeaning fail=0
[2020/09/17-15:54:02.825] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.input.push] T-MbtGxk2T - Using ordering from dataset: {"enabled":false,"rules":[]}
[2020/09/17-15:54:02.826] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.input.push] T-MbtGxk2T - USTP: push selection.method=HEAD_SEQUENTIAL records=100000 ratio=0.02 col=null
[2020/09/17-15:54:02.826] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.push] T-MbtGxk2T - [ct: 36] Push row splits limit= {"maxBytes":-1,"maxRecords":100000,"ordering":{"enabled":false,"rules":[]}}
[2020/09/17-15:54:02.826] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.input.rows] T-MbtGxk2T - [ct: 36] Starting to push split com.dataiku.dip.datasets.sql.AbstractSQLTableDatasetHandler$TableClauseSplit@65a10e70 with limit= 100000
[2020/09/17-15:54:02.827] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 37] Connecting to jdbc:postgresql://localhost:5432/academy with props: {"user":"dkuacademy","password":"***"} conn=postgresql-Y3gqDlO
[2020/09/17-15:54:02.827] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.connections.sql.driver] T-MbtGxk2T - Driver version 9.0
[2020/09/17-15:54:02.829] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 39] Driver: PostgreSQL Native Driver (JDBC 4.0) PostgreSQL 9.0 JDBC4 (build 801) (9.0)
[2020/09/17-15:54:02.830] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 40] Database: PostgreSQL 11.5 (11.5) rowSize=8192 stmts=0
[2020/09/17-15:54:02.830] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.resourceusage] T-MbtGxk2T - [ct: 40] Reporting start of CRU:{"type":"SQL_CONNECTION","id":"wv5SEsbRCGy552Av","startTime":1600350842829,"sqlConnection":{"connection":"postgresql"}}
[2020/09/17-15:54:02.830] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 40] SET MAX ROWS 100000
[2020/09/17-15:54:02.831] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 41] Executing query SELECT * FROM "CREDITCARDFRAUDSTART3_transactions_known"
LIMIT 100000 (st=org.postgresql.jdbc4.Jdbc4Statement@13a4db2c)
[2020/09/17-15:54:02.857] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 67] Done executing query (st=org.postgresql.jdbc4.Jdbc4Statement@13a4db2c)
[2020/09/17-15:54:02.937] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 147] Read 2000 records from DB
[2020/09/17-15:54:03.037] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 247] Read 4000 records from DB
[2020/09/17-15:54:03.149] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 359] Read 6000 records from DB
[2020/09/17-15:54:03.247] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 457] Read 8000 records from DB
[2020/09/17-15:54:03.347] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 557] Read 10000 records from DB
[2020/09/17-15:54:03.445] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 655] Read 12000 records from DB
[2020/09/17-15:54:03.543] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 753] Read 14000 records from DB
[2020/09/17-15:54:03.646] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 856] Read 16000 records from DB
[2020/09/17-15:54:03.745] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 955] Read 18000 records from DB
[2020/09/17-15:54:03.845] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1055] Read 20000 records from DB
[2020/09/17-15:54:03.958] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1168] Read 22000 records from DB
[2020/09/17-15:54:04.056] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1266] Read 24000 records from DB
[2020/09/17-15:54:04.153] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1363] Read 26000 records from DB
[2020/09/17-15:54:04.250] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1460] Read 28000 records from DB
[2020/09/17-15:54:04.349] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1559] Read 30000 records from DB
[2020/09/17-15:54:04.446] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1656] Read 32000 records from DB
[2020/09/17-15:54:04.544] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1754] Read 34000 records from DB
[2020/09/17-15:54:04.646] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1856] Read 36000 records from DB
[2020/09/17-15:54:04.741] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 1951] Read 38000 records from DB
[2020/09/17-15:54:04.840] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2050] Read 40000 records from DB
[2020/09/17-15:54:04.938] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2148] Read 42000 records from DB
[2020/09/17-15:54:05.039] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2249] Read 44000 records from DB
[2020/09/17-15:54:05.136] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2346] Read 46000 records from DB
[2020/09/17-15:54:05.247] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2457] Read 48000 records from DB
[2020/09/17-15:54:05.345] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2555] Read 50000 records from DB
[2020/09/17-15:54:05.448] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2658] Read 52000 records from DB
[2020/09/17-15:54:05.549] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2759] Read 54000 records from DB
[2020/09/17-15:54:05.654] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2864] Read 56000 records from DB
[2020/09/17-15:54:05.754] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 2964] Read 58000 records from DB
[2020/09/17-15:54:05.852] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3062] Read 60000 records from DB
[2020/09/17-15:54:05.950] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3160] Read 62000 records from DB
[2020/09/17-15:54:06.047] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3257] Read 64000 records from DB
[2020/09/17-15:54:06.144] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3354] Read 66000 records from DB
[2020/09/17-15:54:06.241] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3451] Read 68000 records from DB
[2020/09/17-15:54:06.341] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3551] Read 70000 records from DB
[2020/09/17-15:54:06.439] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3649] Read 72000 records from DB
[2020/09/17-15:54:06.546] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3756] Read 74000 records from DB
[2020/09/17-15:54:06.644] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3854] Read 76000 records from DB
[2020/09/17-15:54:06.738] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 3948] Read 78000 records from DB
[2020/09/17-15:54:06.836] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4046] Read 80000 records from DB
[2020/09/17-15:54:06.933] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4143] Read 82000 records from DB
[2020/09/17-15:54:07.032] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4242] Read 84000 records from DB
[2020/09/17-15:54:07.130] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4340] Read 86000 records from DB
[2020/09/17-15:54:07.230] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4440] Read 88000 records from DB
[2020/09/17-15:54:07.330] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4540] Read 90000 records from DB
[2020/09/17-15:54:07.429] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4639] Read 92000 records from DB
[2020/09/17-15:54:07.527] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4737] Read 94000 records from DB
[2020/09/17-15:54:07.626] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4836] Read 96000 records from DB
[2020/09/17-15:54:07.726] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 4936] Read 98000 records from DB
[2020/09/17-15:54:07.835] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 5045] Read 100000 records from DB
[2020/09/17-15:54:07.835] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 5045] Done iterating result set, returned 100000 rows
[2020/09/17-15:54:07.836] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.datasets.sql] T-MbtGxk2T - [ct: 5046] Exited push
[2020/09/17-15:54:07.836] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 5046] Rollback and close conn=postgresql-Y3gqDlO
[2020/09/17-15:54:07.836] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 5046] Rollback conn=postgresql-Y3gqDlO
[2020/09/17-15:54:07.837] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 5047] Close conn=postgresql-Y3gqDlO
[2020/09/17-15:54:07.837] [FT-TrainWorkThread-pTDZ0A63-122050] [DEBUG] [dku.connections.sql.provider] T-MbtGxk2T - [ct: 5047] Close conn=postgresql-Y3gqDlO
[2020/09/17-15:54:07.837] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.resourceusage] T-MbtGxk2T - [ct: 5047] Reporting completion of CRU:{"type":"SQL_CONNECTION","id":"wv5SEsbRCGy552Av","startTime":1600350842829,"sqlConnection":{"connection":"postgresql"}}
[2020/09/17-15:54:07.837] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.input.rows] T-MbtGxk2T - [ct: 5047] Limit done
[2020/09/17-15:54:07.840] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.splits] T-MbtGxk2T - [ct: 5050] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=transactions_known,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 5602e3e1288ac90fc4a90d3faf30b415-0
[2020/09/17-15:54:07.841] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.splits] T-MbtGxk2T - [ct: 5051] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=transactions_known,sel=(method=head-s,records=100000),r=0.8,s=1337 i=5602e3e1288ac90fc4a90d3faf30b415-0
[2020/09/17-15:54:07.842] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.splits] T-MbtGxk2T - [ct: 5052] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=transactions_known,sel=(method=head-s,records=100000),r=0.8,s=1337 i=5602e3e1288ac90fc4a90d3faf30b415-0
[2020/09/17-15:54:07.845] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.ml.python] T-MbtGxk2T - Joining processing thread ...
[2020/09/17-15:54:07.846] [MRT-122051] [INFO] [dku.analysis.ml.python]  - Running a preprocessing set: pp1 in /data/home/dkuacademy/dss/data-dir-design-33333/analysis-data/CREDITCARDFRAUDSTART3/LvXsfpWd/MbtGxk2T/sessions/s1/pp1
[2020/09/17-15:54:07.847] [MRT-122051] [INFO] [dku.block.link]  - Started a socket on port 32820
[2020/09/17-15:54:07.850] [MRT-122051] [INFO] [dku.ml.kernel]  - Writing output of python-single-command-kernel to /data/home/dkuacademy/dss/data-dir-design-33333/analysis-data/CREDITCARDFRAUDSTART3/LvXsfpWd/MbtGxk2T/sessions/s1/pp1/train.log
[2020/09/17-15:54:07.856] [MRT-122051] [INFO] [dku.code.envs.resolution]  - Executing Python activity in builtin env
[2020/09/17-15:54:07.857] [MRT-122051] [WARN] [dku.code.projectLibs]  - External libraries file not found: /data/home/dkuacademy/dss/data-dir-design-33333/config/projects/CREDITCARDFRAUDSTART3/lib/external-libraries.json
[2020/09/17-15:54:07.857] [MRT-122051] [INFO] [dku.code.projectLibs]  - EXTERNAL LIBS FROM CREDITCARDFRAUDSTART3 is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]}
[2020/09/17-15:54:07.858] [MRT-122051] [INFO] [dku.code.projectLibs]  - chunkFolder is /data/home/dkuacademy/dss/data-dir-design-33333/config/projects/CREDITCARDFRAUDSTART3/lib/R
[2020/09/17-15:54:07.858] [MRT-122051] [INFO] [dku.python.single_command.kernel]  - Starting Python process for kernel  python-single-command-kernel
[2020/09/17-15:54:07.858] [MRT-122051] [INFO] [dip.tickets]  - Creating API ticket for analysis-ml-CREDITCARDFRAUDSTART3-USsCBNl on behalf of christelle id=analysis-ml-CREDITCARDFRAUDSTART3-USsCBNl_sbOhfx5qx1gu
[2020/09/17-15:54:07.859] [MRT-122051] [INFO] [dku.security.process]  - Starting process (regular)
[2020/09/17-15:54:07.861] [MRT-122051] [INFO] [dku.security.process]  - Process started with pid=321164
[2020/09/17-15:54:07.861] [MRT-122051] [INFO] [dku.processes.cgroups]  - Will use cgroups []
[2020/09/17-15:54:07.862] [MRT-122051] [INFO] [dku.processes.cgroups]  - Applying rules to used cgroups: []
[2020/09/17-15:54:07.863] [KNL-python-single-command-kernel-monitor-122059] [INFO] [dku.resourceusage]  - Reporting start of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"christelle","projectKey":"CREDITCARDFRAUDSTART3","analysisId":"LvXsfpWd","mlTaskId":"MbtGxk2T","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"vqW6ZLpdt4q4M4b9","startTime":1600350847863,"localProcess":{"cpuCurrent":0.0}}
[2020/09/17-15:54:07.866] [process-resource-monitor-321164-122061] [DEBUG] [dku.resource]  - Process stats for pid 321164: {"pid":321164,"commandName":"/data/home/dkuacademy/dss/data-dir-design-33333/bin/python","cpuUserTimeMS":0,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":0,"cpuCurrent":0.0,"vmSizeMB":18,"vmRSSMB":3,"vmHWMMB":3,"vmRSSAnonMB":1,"vmDataMB":1,"vmSizePeakMB":18,"vmRSSPeakMB":3,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}
Installing debugging signal handler
[2020-09-17 15:54:12,899] [321164/MainThread] [INFO] [root] Connecting to parent at port 32820
[2020-09-17 15:54:12,899] [321164/MainThread] [INFO] [root] Connected to parent at port 32820
[2020/09/17-15:54:12.900] [MRT-122051] [INFO] [dku.link.secret_protected]  - Connected to kernel
[2020/09/17-15:54:12.905] [MRT-122051] [INFO] [dku.block.link.interaction]  - Execute link command respClazz=true respTypeToken=false respIsString=false is=false asyncInputStream=false os=false
[2020-09-17 15:54:16,509] [321164/MainThread] [INFO] [root] Running analysis command: train_prediction_models_nosave
[2020-09-17 15:54:16,510] [321164/MainThread] [INFO] [dataiku.doctor.commands] PPS is {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [{u'mappedValue': 0, u'sourceValue': u'0', u'sampleFreq': 937}, {u'mappedValue': 1, u'sourceValue': u'1', u'sampleFreq': 9063}], u'per_feature': {u'card_first_active_month_parsed': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'Date', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_min': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'signature_provided': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_amount_max': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_age': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_hour': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_ZERO_VARIANCE', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_latitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'days_active': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_year': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_ZERO_VARIANCE', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_location': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_day': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_state_enName': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_cardholder_distance': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_month': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_date_parsed': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'Date', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'transaction_id': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_IDENTIFIER', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_date': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'DateSource', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_latitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_first_active_month': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'DateSource', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'authorized_flag': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_category_id': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_weekend': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_location': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_longitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_id': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_dow': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_max': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'item_category': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_state': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_state': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_id': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_subsector_description': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_longitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_reward_program': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount_min': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_state_enName': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_fico_score': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}}
[2020-09-17 15:54:16,510] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading train set
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [root] Reading with dtypes: None
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <type 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_weekend: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,538] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_cardholder_distance: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,539] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,540] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:16,540] [321164/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'w2_purchase_amount_min': <type 'numpy.float64'>, u'signature_provided': <type 'numpy.float64'>, u'purchase_amount_max': <type 'numpy.float64'>, u'card_age': <type 'numpy.float64'>, u'card_latitude': <type 'numpy.float64'>, u'w2_purchase_amount_avg': <type 'numpy.float64'>, u'purchase_dow': <type 'numpy.float64'>, u'days_active': <type 'numpy.float64'>, u'merchant_location': 'str', u'purchase_date': 'str', u'merchant_latitude': <type 'numpy.float64'>, u'purchase_weekend': <type 'numpy.float64'>, u'purchase_month': <type 'numpy.float64'>, u'card_reward_program': 'str', u'card_state_enName': 'str', u'card_first_active_month': 'str', u'authorized_flag': <type 'numpy.object_'>, u'merchant_category_id': <type 'numpy.float64'>, u'merchant_cardholder_distance': <type 'numpy.float64'>, u'card_location': 'str', u'purchase_amount_avg': <type 'numpy.float64'>, u'merchant_longitude': <type 'numpy.float64'>, u'merchant_id': 'str', u'purchase_day': <type 'numpy.float64'>, u'w2_purchase_amount_max': <type 'numpy.float64'>, u'item_category': 'str', u'card_state': 'str', u'merchant_state': 'str', u'card_id': 'str', u'purchase_amount': <type 'numpy.float64'>, u'merchant_subsector_description': 'str', u'card_longitude': <type 'numpy.float64'>, u'purchase_amount_min': <type 'numpy.float64'>, u'merchant_state_enName': 'str', u'card_fico_score': <type 'numpy.float64'>}
[2020-09-17 15:54:17,366] [321164/MainThread] [INFO] [root] Loaded table
[2020-09-17 15:54:17,368] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2015-09-01
1       2017-11-01
2       2017-10-01
3       2017-01-01
4       2016-08-01
5       2017-06-01
6       2017-06-01
7       2016-03-01
8       2016-12-01
9       2017-07-01
10      2017-09-01
11      2016-07-01
12      2017-10-01
13      2017-05-01
14      2017-09-01
15      2017-07-01
16      2017-01-01
17      2017-01-01
18      2017-10-01
19      2016-12-01
20      2017-08-01
21      2016-08-01
22      2017-09-01
23      2017-08-01
24      2017-08-01
25      2016-11-01
26      2017-09-01
27      2017-10-01
28      2016-03-01
29      2017-08-01
           ...    
79882   2017-02-01
79883   2017-01-01
79884   2015-08-01
79885   2017-03-01
79886   2017-05-01
79887   2016-10-01
79888   2017-02-01
79889   2017-06-01
79890   2017-02-01
79891   2017-02-01
79892   2017-04-01
79893   2016-10-01
79894   2017-03-01
79895   2016-10-01
79896   2017-05-01
79897   2014-05-01
79898   2017-01-01
79899   2017-02-01
79900   2016-12-01
79901   2016-08-01
79902   2017-06-01
79903   2017-04-01
79904   2017-07-01
79905   2016-11-01
79906   2017-02-01
79907   2016-02-01
79908   2016-08-01
79909   2015-04-01
79910   2016-01-01
79911   2017-01-01
Name: card_first_active_month_parsed, Length: 79912, dtype: datetime64[ns]
[2020-09-17 15:54:17,372] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.650054e+09
1        3.718483e+09
2        3.715805e+09
3        3.692218e+09
4        3.678998e+09
5        3.705264e+09
6        3.705264e+09
7        3.665779e+09
8        3.689539e+09
9        3.707856e+09
10       3.713213e+09
11       3.676320e+09
12       3.715805e+09
13       3.702586e+09
14       3.713213e+09
15       3.707856e+09
16       3.692218e+09
17       3.692218e+09
18       3.715805e+09
19       3.689539e+09
20       3.710534e+09
21       3.678998e+09
22       3.713213e+09
23       3.710534e+09
24       3.710534e+09
25       3.686947e+09
26       3.713213e+09
27       3.715805e+09
28       3.665779e+09
29       3.710534e+09
             ...     
79882    3.694896e+09
79883    3.692218e+09
79884    3.647376e+09
79885    3.697315e+09
79886    3.702586e+09
79887    3.684269e+09
79888    3.694896e+09
79889    3.705264e+09
79890    3.694896e+09
79891    3.694896e+09
79892    3.699994e+09
79893    3.684269e+09
79894    3.697315e+09
79895    3.684269e+09
79896    3.702586e+09
79897    3.607891e+09
79898    3.692218e+09
79899    3.694896e+09
79900    3.689539e+09
79901    3.678998e+09
79902    3.705264e+09
79903    3.699994e+09
79904    3.707856e+09
79905    3.686947e+09
79906    3.694896e+09
79907    3.663274e+09
79908    3.678998e+09
79909    3.636835e+09
79910    3.660595e+09
79911    3.692218e+09
Name: card_first_active_month_parsed, Length: 79912, dtype: float64
[2020-09-17 15:54:17,375] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-11-08
1       2017-11-08
2       2017-11-08
3       2017-11-08
4       2017-11-08
5       2017-11-08
6       2017-11-08
7       2017-11-08
8       2017-11-08
9       2017-11-08
10      2017-11-08
11      2017-11-08
12      2017-11-08
13      2017-11-08
14      2017-11-08
15      2017-11-08
16      2017-11-08
17      2017-11-08
18      2017-11-08
19      2017-11-08
20      2017-11-08
21      2017-11-08
22      2017-11-08
23      2017-11-08
24      2017-11-08
25      2017-11-08
26      2017-11-08
27      2017-11-08
28      2017-11-08
29      2017-11-08
           ...    
79882   2017-06-11
79883   2017-06-14
79884   2017-06-21
79885   2017-06-22
79886   2017-06-22
79887   2017-06-23
79888   2017-06-26
79889   2017-06-27
79890   2017-06-29
79891   2017-07-04
79892   2017-07-06
79893   2017-07-07
79894   2017-07-07
79895   2017-07-09
79896   2017-07-10
79897   2017-07-13
79898   2017-07-20
79899   2017-07-25
79900   2017-07-26
79901   2017-07-26
79902   2017-08-01
79903   2017-08-02
79904   2017-08-03
79905   2017-08-05
79906   2017-08-05
79907   2017-08-06
79908   2017-08-08
79909   2017-08-10
79910   2017-08-16
79911   2017-08-17
Name: purchase_date_parsed, Length: 79912, dtype: datetime64[ns]
[2020-09-17 15:54:17,378] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.719088e+09
1        3.719088e+09
2        3.719088e+09
3        3.719088e+09
4        3.719088e+09
5        3.719088e+09
6        3.719088e+09
7        3.719088e+09
8        3.719088e+09
9        3.719088e+09
10       3.719088e+09
11       3.719088e+09
12       3.719088e+09
13       3.719088e+09
14       3.719088e+09
15       3.719088e+09
16       3.719088e+09
17       3.719088e+09
18       3.719088e+09
19       3.719088e+09
20       3.719088e+09
21       3.719088e+09
22       3.719088e+09
23       3.719088e+09
24       3.719088e+09
25       3.719088e+09
26       3.719088e+09
27       3.719088e+09
28       3.719088e+09
29       3.719088e+09
             ...     
79882    3.706128e+09
79883    3.706387e+09
79884    3.706992e+09
79885    3.707078e+09
79886    3.707078e+09
79887    3.707165e+09
79888    3.707424e+09
79889    3.707510e+09
79890    3.707683e+09
79891    3.708115e+09
79892    3.708288e+09
79893    3.708374e+09
79894    3.708374e+09
79895    3.708547e+09
79896    3.708634e+09
79897    3.708893e+09
79898    3.709498e+09
79899    3.709930e+09
79900    3.710016e+09
79901    3.710016e+09
79902    3.710534e+09
79903    3.710621e+09
79904    3.710707e+09
79905    3.710880e+09
79906    3.710880e+09
79907    3.710966e+09
79908    3.711139e+09
79909    3.711312e+09
79910    3.711830e+09
79911    3.711917e+09
Name: purchase_date_parsed, Length: 79912, dtype: float64
[2020-09-17 15:54:17,379] [321164/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2020-09-17 15:54:17,379] [321164/MainThread] [INFO] [dataiku.doctor.commands] Loaded train df: shape=(79912,40)
[2020-09-17 15:54:17,379] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : transaction_id (int64)
[2020-09-17 15:54:17,379] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : authorized_flag (object)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_date (object)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_date_parsed (float64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_year (int64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_month (float64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_day (float64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_dow (float64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_weekend (float64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_hour (int64)
[2020-09-17 15:54:17,380] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_id (object)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_id (object)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_category_id (float64)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : item_category (object)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_amount (float64)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : signature_provided (float64)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_first_active_month (object)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_first_active_month_parsed (float64)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : days_active (float64)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_reward_program (object)
[2020-09-17 15:54:17,381] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_latitude (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_longitude (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_fico_score (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_age (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_subsector_description (object)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_latitude (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_longitude (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_location (object)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_cardholder_distance (float64)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_state (object)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_state_enName (object)
[2020-09-17 15:54:17,382] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_location (object)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_state (object)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_state_enName (object)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_amount_min (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_amount_max (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_amount_avg (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : w2_purchase_amount_min (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : w2_purchase_amount_max (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.commands] Train col : w2_purchase_amount_avg (float64)
[2020-09-17 15:54:17,383] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading train set
[2020-09-17 15:54:17,384] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading test set
[2020-09-17 15:54:17,429] [321164/MainThread] [INFO] [root] Reading with dtypes: None
[2020-09-17 15:54:17,429] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <type 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_weekend: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,430] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_cardholder_distance: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,431] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:54:17,432] [321164/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'w2_purchase_amount_min': <type 'numpy.float64'>, u'signature_provided': <type 'numpy.float64'>, u'purchase_amount_max': <type 'numpy.float64'>, u'card_age': <type 'numpy.float64'>, u'card_latitude': <type 'numpy.float64'>, u'w2_purchase_amount_avg': <type 'numpy.float64'>, u'purchase_dow': <type 'numpy.float64'>, u'days_active': <type 'numpy.float64'>, u'merchant_location': 'str', u'purchase_date': 'str', u'merchant_latitude': <type 'numpy.float64'>, u'purchase_weekend': <type 'numpy.float64'>, u'purchase_month': <type 'numpy.float64'>, u'card_reward_program': 'str', u'card_state_enName': 'str', u'card_first_active_month': 'str', u'authorized_flag': <type 'numpy.object_'>, u'merchant_category_id': <type 'numpy.float64'>, u'merchant_cardholder_distance': <type 'numpy.float64'>, u'card_location': 'str', u'purchase_amount_avg': <type 'numpy.float64'>, u'merchant_longitude': <type 'numpy.float64'>, u'merchant_id': 'str', u'purchase_day': <type 'numpy.float64'>, u'w2_purchase_amount_max': <type 'numpy.float64'>, u'item_category': 'str', u'card_state': 'str', u'merchant_state': 'str', u'card_id': 'str', u'purchase_amount': <type 'numpy.float64'>, u'merchant_subsector_description': 'str', u'card_longitude': <type 'numpy.float64'>, u'purchase_amount_min': <type 'numpy.float64'>, u'merchant_state_enName': 'str', u'card_fico_score': <type 'numpy.float64'>}
[2020-09-17 15:54:17,744] [321164/MainThread] [INFO] [root] Loaded table
[2020-09-17 15:54:17,746] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-01-01
1       2015-11-01
2       2017-10-01
3       2017-06-01
4       2016-04-01
5       2017-05-01
6       2016-12-01
7       2016-12-01
8       2017-11-01
9       2017-10-01
10      2016-02-01
11      2014-09-01
12      2014-08-01
13      2017-02-01
14      2016-12-01
15      2017-06-01
16      2017-07-01
17      2017-06-01
18      2017-06-01
19      2017-10-01
20      2017-10-01
21      2016-03-01
22      2017-05-01
23      2016-12-01
24      2017-07-01
25      2016-09-01
26      2017-10-01
27      2017-10-01
28      2016-06-01
29      2015-07-01
           ...    
20058   2017-06-01
20059   2015-04-01
20060   2017-01-01
20061   2017-10-01
20062   2016-07-01
20063   2017-01-01
20064   2017-06-01
20065   2017-07-01
20066   2017-04-01
20067   2017-04-01
20068   2016-08-01
20069   2017-02-01
20070   2017-01-01
20071   2016-02-01
20072   2017-06-01
20073   2017-02-01
20074   2016-11-01
20075   2017-03-01
20076   2016-06-01
20077   2016-06-01
20078   2017-02-01
20079   2017-03-01
20080   2017-06-01
20081   2016-11-01
20082   2017-02-01
20083   2017-01-01
20084   2016-11-01
20085   2016-10-01
20086   2017-05-01
20087   2017-06-01
Name: card_first_active_month_parsed, Length: 20088, dtype: datetime64[ns]
[2020-09-17 15:54:17,750] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.692218e+09
1        3.655325e+09
2        3.715805e+09
3        3.705264e+09
4        3.668458e+09
5        3.702586e+09
6        3.689539e+09
7        3.689539e+09
8        3.718483e+09
9        3.715805e+09
10       3.663274e+09
11       3.618518e+09
12       3.615840e+09
13       3.694896e+09
14       3.689539e+09
15       3.705264e+09
16       3.707856e+09
17       3.705264e+09
18       3.705264e+09
19       3.715805e+09
20       3.715805e+09
21       3.665779e+09
22       3.702586e+09
23       3.689539e+09
24       3.707856e+09
25       3.681677e+09
26       3.715805e+09
27       3.715805e+09
28       3.673728e+09
29       3.644698e+09
             ...     
20058    3.705264e+09
20059    3.636835e+09
20060    3.692218e+09
20061    3.715805e+09
20062    3.676320e+09
20063    3.692218e+09
20064    3.705264e+09
20065    3.707856e+09
20066    3.699994e+09
20067    3.699994e+09
20068    3.678998e+09
20069    3.694896e+09
20070    3.692218e+09
20071    3.663274e+09
20072    3.705264e+09
20073    3.694896e+09
20074    3.686947e+09
20075    3.697315e+09
20076    3.673728e+09
20077    3.673728e+09
20078    3.694896e+09
20079    3.697315e+09
20080    3.705264e+09
20081    3.686947e+09
20082    3.694896e+09
20083    3.692218e+09
20084    3.686947e+09
20085    3.684269e+09
20086    3.702586e+09
20087    3.705264e+09
Name: card_first_active_month_parsed, Length: 20088, dtype: float64
[2020-09-17 15:54:17,754] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-11-08
1       2017-11-08
2       2017-11-08
3       2017-11-08
4       2017-11-08
5       2017-11-08
6       2017-11-08
7       2017-11-08
8       2017-11-08
9       2017-11-08
10      2017-11-09
11      2017-11-09
12      2017-11-09
13      2017-11-09
14      2017-11-09
15      2017-11-09
16      2017-11-09
17      2017-11-09
18      2017-11-09
19      2017-11-09
20      2017-11-09
21      2017-11-09
22      2017-11-10
23      2017-11-10
24      2017-11-10
25      2017-11-10
26      2017-11-10
27      2017-11-10
28      2017-11-10
29      2017-11-10
           ...    
20058   2017-07-14
20059   2017-08-10
20060   2017-08-11
20061   2017-10-18
20062   2017-10-28
20063   2017-11-22
20064   2017-11-24
20065   2017-12-20
20066   2017-08-06
20067   2017-04-26
20068   2017-06-09
20069   2017-07-07
20070   2017-01-31
20071   2017-02-17
20072   2017-09-19
20073   2017-12-08
20074   2017-03-17
20075   2017-04-04
20076   2017-05-03
20077   2017-05-07
20078   2017-06-10
20079   2017-06-22
20080   2017-06-28
20081   2017-07-24
20082   2017-07-29
20083   2017-08-01
20084   2017-08-02
20085   2017-08-07
20086   2017-08-15
20087   2017-08-16
Name: purchase_date_parsed, Length: 20088, dtype: datetime64[ns]
[2020-09-17 15:54:17,758] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.719088e+09
1        3.719088e+09
2        3.719088e+09
3        3.719088e+09
4        3.719088e+09
5        3.719088e+09
6        3.719088e+09
7        3.719088e+09
8        3.719088e+09
9        3.719088e+09
10       3.719174e+09
11       3.719174e+09
12       3.719174e+09
13       3.719174e+09
14       3.719174e+09
15       3.719174e+09
16       3.719174e+09
17       3.719174e+09
18       3.719174e+09
19       3.719174e+09
20       3.719174e+09
21       3.719174e+09
22       3.719261e+09
23       3.719261e+09
24       3.719261e+09
25       3.719261e+09
26       3.719261e+09
27       3.719261e+09
28       3.719261e+09
29       3.719261e+09
             ...     
20058    3.708979e+09
20059    3.711312e+09
20060    3.711398e+09
20061    3.717274e+09
20062    3.718138e+09
20063    3.720298e+09
20064    3.720470e+09
20065    3.722717e+09
20066    3.710966e+09
20067    3.702154e+09
20068    3.705955e+09
20069    3.708374e+09
20070    3.694810e+09
20071    3.696278e+09
20072    3.714768e+09
20073    3.721680e+09
20074    3.698698e+09
20075    3.700253e+09
20076    3.702758e+09
20077    3.703104e+09
20078    3.706042e+09
20079    3.707078e+09
20080    3.707597e+09
20081    3.709843e+09
20082    3.710275e+09
20083    3.710534e+09
20084    3.710621e+09
20085    3.711053e+09
20086    3.711744e+09
20087    3.711830e+09
Name: purchase_date_parsed, Length: 20088, dtype: float64
[2020-09-17 15:54:17,759] [321164/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2020-09-17 15:54:17,759] [321164/MainThread] [INFO] [dataiku.doctor.commands] Loaded test df: shape=(20088,40)
[2020-09-17 15:54:17,760] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading test set
[2020-09-17 15:54:17,760] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Collecting statistics
[2020-09-17 15:54:17,760] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_first_active_month_parsed... (type=NUMERIC)
[2020-09-17 15:54:17,760] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,765] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at w2_purchase_amount_min... (type=NUMERIC)
[2020-09-17 15:54:17,765] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,770] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at signature_provided... (type=NUMERIC)
[2020-09-17 15:54:17,770] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,773] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_amount_max... (type=NUMERIC)
[2020-09-17 15:54:17,774] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,778] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_age... (type=NUMERIC)
[2020-09-17 15:54:17,778] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,782] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_hour... (type=NUMERIC)
[2020-09-17 15:54:17,782] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_latitude... (type=NUMERIC)
[2020-09-17 15:54:17,782] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,786] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at w2_purchase_amount_avg... (type=NUMERIC)
[2020-09-17 15:54:17,787] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,791] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at days_active... (type=NUMERIC)
[2020-09-17 15:54:17,791] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,795] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_year... (type=NUMERIC)
[2020-09-17 15:54:17,795] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_location... (type=CATEGORY)
[2020-09-17 15:54:17,836] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_day... (type=NUMERIC)
[2020-09-17 15:54:17,836] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,840] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_state_enName... (type=CATEGORY)
[2020-09-17 15:54:17,864] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_cardholder_distance... (type=NUMERIC)
[2020-09-17 15:54:17,864] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,868] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_month... (type=NUMERIC)
[2020-09-17 15:54:17,869] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,873] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_date_parsed... (type=NUMERIC)
[2020-09-17 15:54:17,873] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,877] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at transaction_id... (type=NUMERIC)
[2020-09-17 15:54:17,877] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_date... (type=CATEGORY)
[2020-09-17 15:54:17,901] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_latitude... (type=NUMERIC)
[2020-09-17 15:54:17,901] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,905] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_first_active_month... (type=CATEGORY)
[2020-09-17 15:54:17,927] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at authorized_flag... (type=NUMERIC)
[2020-09-17 15:54:17,928] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_category_id... (type=NUMERIC)
[2020-09-17 15:54:17,928] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,932] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_weekend... (type=NUMERIC)
[2020-09-17 15:54:17,932] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,936] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_location... (type=CATEGORY)
[2020-09-17 15:54:17,988] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_amount_avg... (type=NUMERIC)
[2020-09-17 15:54:17,988] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,992] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_longitude... (type=NUMERIC)
[2020-09-17 15:54:17,993] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:17,996] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_id... (type=CATEGORY)
[2020-09-17 15:54:18,022] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_dow... (type=NUMERIC)
[2020-09-17 15:54:18,022] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,025] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at w2_purchase_amount_max... (type=NUMERIC)
[2020-09-17 15:54:18,025] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,028] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at item_category... (type=CATEGORY)
[2020-09-17 15:54:18,041] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_state... (type=CATEGORY)
[2020-09-17 15:54:18,055] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_state... (type=CATEGORY)
[2020-09-17 15:54:18,069] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_id... (type=CATEGORY)
[2020-09-17 15:54:18,100] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_amount... (type=NUMERIC)
[2020-09-17 15:54:18,100] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,103] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_subsector_description... (type=CATEGORY)
[2020-09-17 15:54:18,117] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_longitude... (type=NUMERIC)
[2020-09-17 15:54:18,117] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,120] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_reward_program... (type=CATEGORY)
[2020-09-17 15:54:18,134] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_amount_min... (type=NUMERIC)
[2020-09-17 15:54:18,134] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,138] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_state_enName... (type=CATEGORY)
[2020-09-17 15:54:18,151] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_fico_score... (type=NUMERIC)
[2020-09-17 15:54:18,151] [321164/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2020-09-17 15:54:18,156] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Collecting statistics
[2020-09-17 15:54:18,157] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] generating interactions
[2020-09-17 15:54:18,158] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [{u'mappedValue': 0, u'sourceValue': u'0', u'sampleFreq': 937}, {u'mappedValue': 1, u'sourceValue': u'1', u'sampleFreq': 9063}], u'per_feature': {u'card_first_active_month_parsed': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'Date', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_min': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'signature_provided': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_amount_max': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_age': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_hour': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_ZERO_VARIANCE', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_latitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'days_active': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_year': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_ZERO_VARIANCE', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_location': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_day': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_state_enName': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_cardholder_distance': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_month': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_date_parsed': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'Date', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'transaction_id': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'REJECT', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'autoReason': u'REJECT_IDENTIFIER', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_date': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'DateSource', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_latitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_first_active_month': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'DateSource', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'authorized_flag': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_category_id': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'purchase_weekend': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_location': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_longitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_id': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_dow': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'w2_purchase_amount_max': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'item_category': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_state': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'merchant_state': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_id': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_subsector_description': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_longitude': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'card_reward_program': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'purchase_amount_min': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'merchant_state_enName': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'USStateMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'card_fico_score': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}}
[2020-09-17 15:54:18,158] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] No feature selection to perform
[2020-09-17 15:54:18,158] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing train set
[2020-09-17 15:54:18,159] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 79912
[2020-09-17 15:54:18,159] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RemapValueToOutput
[2020-09-17 15:54:18,170] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,171] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {u'card_first_active_month_parsed': 3688143513.264591, u'w2_purchase_amount_min': 83.78822567324104, u'signature_provided': 0.1883947342076284, u'purchase_amount_max': 712.0422547301954, u'card_age': 45.787779056962655, u'card_latitude': 39.58343451546729, u'w2_purchase_amount_avg': 253.1698966305378, u'days_active': 268.1083316648313, u'purchase_day': 16.013414756231857, u'merchant_latitude': 39.600259810791506, u'purchase_weekend': 0.28363700070077086, u'purchase_month': 7.795487536289919, u'purchase_date_parsed': 3711308073.1204324, u'merchant_category_id': 389.461395034538, u'merchant_cardholder_distance': 999.9677089798583, u'purchase_amount_avg': 253.4390651336389, u'merchant_longitude': -99.92398076634855, u'purchase_dow': 4.03627740514566, u'w2_purchase_amount_max': 1550.574005906551, u'purchase_amount': 253.87922952745205, u'card_longitude': -98.94784431624568, u'purchase_amount_min': 89.26516030133085, u'card_fico_score': 572.2911577735509}
[2020-09-17 15:54:18,187] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_first_active_month_parsed)
[2020-09-17 15:54:18,188] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_first_active_month_parsed (avg=3688143513.26 std=23343374.5418 shift=3688143513.26 inv_scale=4.28387077545e-08)
[2020-09-17 15:54:18,208] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_first_active_month_parsed (avg=-7.90871877114e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,208] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_min)
[2020-09-17 15:54:18,208] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_min (avg=83.7882256732 std=399.613622833 shift=83.7882256732 inv_scale=0.00250241719216)
[2020-09-17 15:54:18,210] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_min (avg=-2.95823823608e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,211] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (signature_provided)
[2020-09-17 15:54:18,211] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale signature_provided (avg=0.188394734208 std=0.391029502386 shift=0.188394734208 inv_scale=2.55735179545)
[2020-09-17 15:54:18,213] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled signature_provided (avg=-3.00270370186e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,213] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_amount_max)
[2020-09-17 15:54:18,214] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_max (avg=712.04225473 std=1361.38801887 shift=712.04225473 inv_scale=0.000734544440041)
[2020-09-17 15:54:18,216] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_max (avg=5.78650263266e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,216] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_age)
[2020-09-17 15:54:18,217] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_age (avg=45.787779057 std=20.6197455193 shift=45.787779057 inv_scale=0.0484972037633)
[2020-09-17 15:54:18,219] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_age (avg=2.16348446149e-16 std=1.0) nulls=0
[2020-09-17 15:54:18,219] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_latitude)
[2020-09-17 15:54:18,219] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_latitude (avg=39.5834345155 std=5.32426523674 shift=39.5834345155 inv_scale=0.18781934324)
[2020-09-17 15:54:18,222] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_latitude (avg=-5.16583194503e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,222] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_avg)
[2020-09-17 15:54:18,222] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_avg (avg=253.169896631 std=487.138223449 shift=253.169896631 inv_scale=0.00205280545)
[2020-09-17 15:54:18,224] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_avg (avg=2.19106178902e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,225] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (days_active)
[2020-09-17 15:54:18,225] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale days_active (avg=268.108331665 std=254.629471048 shift=268.108331665 inv_scale=0.00392727517316)
[2020-09-17 15:54:18,227] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled days_active (avg=-6.05119618444e-17 std=1.0) nulls=0
[2020-09-17 15:54:18,227] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_day)
[2020-09-17 15:54:18,228] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_day (avg=16.0134147562 std=8.69515420167 shift=16.0134147562 inv_scale=0.11500658606)
[2020-09-17 15:54:18,230] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_day (avg=5.43792133379e-17 std=1.0) nulls=0
[2020-09-17 15:54:18,230] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (merchant_cardholder_distance)
[2020-09-17 15:54:18,231] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_cardholder_distance (avg=999.96770898 std=540.011349513 shift=999.96770898 inv_scale=0.00185181293116)
[2020-09-17 15:54:18,233] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_cardholder_distance (avg=3.61910338185e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,233] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_month)
[2020-09-17 15:54:18,233] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_month (avg=7.79548753629 std=3.39911929747 shift=7.79548753629 inv_scale=0.294193852138)
[2020-09-17 15:54:18,236] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_month (avg=-9.51366742585e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,236] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_date_parsed)
[2020-09-17 15:54:18,236] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_date_parsed (avg=3711308073.12 std=8988317.59225 shift=3711308073.12 inv_scale=1.11255525824e-07)
[2020-09-17 15:54:18,238] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_date_parsed (avg=1.20857618429e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,239] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (merchant_latitude)
[2020-09-17 15:54:18,239] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_latitude (avg=39.6002598108 std=5.21060625154 shift=39.6002598108 inv_scale=0.191916247693)
[2020-09-17 15:54:18,241] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_latitude (avg=6.96956285634e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,241] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (merchant_category_id)
[2020-09-17 15:54:18,242] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_category_id (avg=389.461395035 std=226.93114766 shift=389.461395035 inv_scale=0.00440662293524)
[2020-09-17 15:54:18,244] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_category_id (avg=-5.51721603187e-17 std=1.0) nulls=0
[2020-09-17 15:54:18,244] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_weekend)
[2020-09-17 15:54:18,244] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_weekend (avg=0.283637000701 std=0.450765565679 shift=0.283637000701 inv_scale=2.21844807177)
[2020-09-17 15:54:18,247] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_weekend (avg=-2.79982319795e-15 std=0.999999999999) nulls=0
[2020-09-17 15:54:18,247] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_amount_avg)
[2020-09-17 15:54:18,247] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_avg (avg=253.439065134 std=475.421221546 shift=253.439065134 inv_scale=0.00210339790207)
[2020-09-17 15:54:18,249] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_avg (avg=3.98607743847e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,250] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (merchant_longitude)
[2020-09-17 15:54:18,250] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_longitude (avg=-99.9239807663 std=13.6329202736 shift=-99.9239807663 inv_scale=0.07335185565)
[2020-09-17 15:54:18,252] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_longitude (avg=4.08665159643e-13 std=1.0) nulls=0
[2020-09-17 15:54:18,252] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_dow)
[2020-09-17 15:54:18,253] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_dow (avg=4.03627740515 std=1.91791948068 shift=4.03627740515 inv_scale=0.521398322544)
[2020-09-17 15:54:18,255] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_dow (avg=-9.90775964402e-16 std=1.0) nulls=0
[2020-09-17 15:54:18,255] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_max)
[2020-09-17 15:54:18,255] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_max (avg=1550.57400591 std=4221.62537248 shift=1550.57400591 inv_scale=0.000236875589795)
[2020-09-17 15:54:18,257] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_max (avg=-1.3337697483e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,257] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_amount)
[2020-09-17 15:54:18,258] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount (avg=253.879229527 std=678.063183443 shift=253.879229527 inv_scale=0.00147478881676)
[2020-09-17 15:54:18,260] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount (avg=4.18141789533e-14 std=1.0) nulls=0
[2020-09-17 15:54:18,260] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_longitude)
[2020-09-17 15:54:18,260] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_longitude (avg=-98.9478443162 std=13.6286754697 shift=-98.9478443162 inv_scale=0.0733747019088)
[2020-09-17 15:54:18,262] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_longitude (avg=-1.61024277582e-13 std=1.0) nulls=0
[2020-09-17 15:54:18,263] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_amount_min)
[2020-09-17 15:54:18,263] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_min (avg=89.2651603013 std=368.879906097 shift=89.2651603013 inv_scale=0.00271090938669)
[2020-09-17 15:54:18,266] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_min (avg=1.60002599261e-15 std=1.0) nulls=0
[2020-09-17 15:54:18,266] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_fico_score)
[2020-09-17 15:54:18,267] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_fico_score (avg=572.291157774 std=169.60312568 shift=572.291157774 inv_scale=0.00589611775131)
[2020-09-17 15:54:18,270] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_fico_score (avg=-1.6205710648e-16 std=1.0) nulls=0
[2020-09-17 15:54:18,270] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2020-09-17 15:54:18,270] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_location)
[2020-09-17 15:54:18,298] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 103) nnz=79912
[2020-09-17 15:54:18,299] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_state_enName)
[2020-09-17 15:54:18,331] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 48) nnz=79912
[2020-09-17 15:54:18,332] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (purchase_date)
[2020-09-17 15:54:18,361] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 103) nnz=79912
[2020-09-17 15:54:18,362] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_first_active_month)
[2020-09-17 15:54:18,392] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 74) nnz=79912
[2020-09-17 15:54:18,393] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_location)
[2020-09-17 15:54:18,424] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 103) nnz=79912
[2020-09-17 15:54:18,425] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_id)
[2020-09-17 15:54:18,454] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 103) nnz=79912
[2020-09-17 15:54:18,455] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (item_category)
[2020-09-17 15:54:18,482] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 6) nnz=79912
[2020-09-17 15:54:18,483] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_state)
[2020-09-17 15:54:18,515] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 48) nnz=79912
[2020-09-17 15:54:18,516] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state)
[2020-09-17 15:54:18,547] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 48) nnz=79912
[2020-09-17 15:54:18,548] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_id)
[2020-09-17 15:54:18,577] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 103) nnz=79912
[2020-09-17 15:54:18,578] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_subsector_description)
[2020-09-17 15:54:18,607] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 40) nnz=79912
[2020-09-17 15:54:18,609] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (card_reward_program)
[2020-09-17 15:54:18,640] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 5) nnz=79912
[2020-09-17 15:54:18,641] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state_enName)
[2020-09-17 15:54:18,676] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 48) nnz=79912
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(interaction)
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RealignTarget
[2020-09-17 15:54:18,677] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (79912,)
[2020-09-17 15:54:18,681] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (79912,)
[2020-09-17 15:54:18,681] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DropRowsWhereNoTarget
[2020-09-17 15:54:18,683] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because no target
[2020-09-17 15:54:18,683] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (79912, 855) target before = (79912,)
[2020-09-17 15:54:18,689] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(79912, 40)
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (79912, 855) target after = (79912,)
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (79912, 40) 
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(79912, 855) 
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((79912,))
[2020-09-17 15:54:18,783] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:EmitCurrentMFAsResult
[2020-09-17 15:54:18,784] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 79912
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (79912, 40) 
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((79912, 40))
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((79912, 855))
[2020-09-17 15:54:18,784] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((79912,))
[2020-09-17 15:54:18,787] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing train set
[2020-09-17 15:54:18,788] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing test set
[2020-09-17 15:54:18,816] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 20088
[2020-09-17 15:54:18,816] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RemapValueToOutput
[2020-09-17 15:54:18,820] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,820] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {u'card_first_active_month_parsed': 3688143513.264591, u'w2_purchase_amount_min': 83.78822567324104, u'signature_provided': 0.1883947342076284, u'purchase_amount_max': 712.0422547301954, u'card_age': 45.787779056962655, u'card_latitude': 39.58343451546729, u'w2_purchase_amount_avg': 253.1698966305378, u'days_active': 268.1083316648313, u'purchase_day': 16.013414756231857, u'merchant_latitude': 39.600259810791506, u'purchase_weekend': 0.28363700070077086, u'purchase_month': 7.795487536289919, u'purchase_date_parsed': 3711308073.1204324, u'merchant_category_id': 389.461395034538, u'merchant_cardholder_distance': 999.9677089798583, u'purchase_amount_avg': 253.4390651336389, u'merchant_longitude': -99.92398076634855, u'purchase_dow': 4.03627740514566, u'w2_purchase_amount_max': 1550.574005906551, u'purchase_amount': 253.87922952745205, u'card_longitude': -98.94784431624568, u'purchase_amount_min': 89.26516030133085, u'card_fico_score': 572.2911577735509}
[2020-09-17 15:54:18,829] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_first_active_month_parsed)
[2020-09-17 15:54:18,830] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_first_active_month_parsed (avg=3688153122.58 std=23429599.9149 shift=3688143513.26 inv_scale=4.28387077545e-08)
[2020-09-17 15:54:18,831] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_first_active_month_parsed (avg=0.000411650682157 std=1.00369378356) nulls=0
[2020-09-17 15:54:18,832] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_min)
[2020-09-17 15:54:18,832] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_min (avg=84.3180854241 std=538.625666457 shift=83.7882256732 inv_scale=0.00250241719216)
[2020-09-17 15:54:18,834] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_min (avg=0.00132593015007 std=1.34786612788) nulls=0
[2020-09-17 15:54:18,834] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (signature_provided)
[2020-09-17 15:54:18,834] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale signature_provided (avg=0.191557148546 std=0.393536169876 shift=0.188394734208 inv_scale=2.55735179545)
[2020-09-17 15:54:18,836] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled signature_provided (avg=0.0080874059872 std=1.00641043061) nulls=0
[2020-09-17 15:54:18,836] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_amount_max)
[2020-09-17 15:54:18,836] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_max (avg=712.117355635 std=1317.70245922 shift=712.04225473 inv_scale=0.000734544440041)
[2020-09-17 15:54:18,838] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_max (avg=5.51649522169e-05 std=0.967911015052) nulls=0
[2020-09-17 15:54:18,838] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_age)
[2020-09-17 15:54:18,838] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_age (avg=45.7579649542 std=20.5356933734 shift=45.787779057 inv_scale=0.0484972037633)
[2020-09-17 15:54:18,840] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_age (avg=-0.00144590061663 std=0.99592370595) nulls=0
[2020-09-17 15:54:18,840] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_latitude)
[2020-09-17 15:54:18,841] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_latitude (avg=39.6108516527 std=5.32010138829 shift=39.5834345155 inv_scale=0.18781934324)
[2020-09-17 15:54:18,842] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_latitude (avg=0.00514946871382 std=0.999217948718) nulls=0
[2020-09-17 15:54:18,842] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_avg)
[2020-09-17 15:54:18,843] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_avg (avg=253.366570442 std=605.48575123 shift=253.169896631 inv_scale=0.00205280545)
[2020-09-17 15:54:18,844] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_avg (avg=0.000403733071633 std=1.24294445002) nulls=0
[2020-09-17 15:54:18,844] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (days_active)
[2020-09-17 15:54:18,845] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale days_active (avg=267.363201912 std=255.323490371 shift=268.108331665 inv_scale=0.00392727517316)
[2020-09-17 15:54:18,846] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled days_active (avg=-0.00292632958069 std=1.00272560486) nulls=0
[2020-09-17 15:54:18,846] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_day)
[2020-09-17 15:54:18,847] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_day (avg=15.9050179211 std=8.7022764197 shift=16.0134147562 inv_scale=0.11500658606)
[2020-09-17 15:54:18,848] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_day (avg=-0.0124663499428 std=1.00081910198) nulls=0
[2020-09-17 15:54:18,848] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (merchant_cardholder_distance)
[2020-09-17 15:54:18,848] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_cardholder_distance (avg=1003.90753485 std=542.268873831 shift=999.96770898 inv_scale=0.00185181293116)
[2020-09-17 15:54:18,850] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_cardholder_distance (avg=0.00729582048668 std=1.00418051272) nulls=0
[2020-09-17 15:54:18,850] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_month)
[2020-09-17 15:54:18,850] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_month (avg=7.77847471127 std=3.39607723582 shift=7.79548753629 inv_scale=0.294193852138)
[2020-09-17 15:54:18,852] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_month (avg=-0.00500506852824 std=0.999105044167) nulls=0
[2020-09-17 15:54:18,852] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_date_parsed)
[2020-09-17 15:54:18,852] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_date_parsed (avg=3711253303.23 std=8980458.85574 shift=3711308073.12 inv_scale=1.11255525824e-07)
[2020-09-17 15:54:18,854] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_date_parsed (avg=-0.00609345342594 std=0.999125672138) nulls=0
[2020-09-17 15:54:18,854] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (merchant_latitude)
[2020-09-17 15:54:18,854] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_latitude (avg=39.5615785544 std=5.22503505033 shift=39.6002598108 inv_scale=0.191916247693)
[2020-09-17 15:54:18,856] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_latitude (avg=-0.00742356159022 std=1.00276912092) nulls=0
[2020-09-17 15:54:18,856] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (merchant_category_id)
[2020-09-17 15:54:18,856] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_category_id (avg=388.464058144 std=226.766975805 shift=389.461395035 inv_scale=0.00440662293524)
[2020-09-17 15:54:18,857] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_category_id (avg=-0.00439488761527 std=0.999276556537) nulls=0
[2020-09-17 15:54:18,857] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_weekend)
[2020-09-17 15:54:18,858] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_weekend (avg=0.281212664277 std=0.449602229264 shift=0.283637000701 inv_scale=2.21844807177)
[2020-09-17 15:54:18,859] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_weekend (avg=-0.00537826446424 std=0.997419198574) nulls=0
[2020-09-17 15:54:18,859] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_amount_avg)
[2020-09-17 15:54:18,860] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_avg (avg=250.73487527 std=452.059991439 shift=253.439065134 inv_scale=0.00210339790207)
[2020-09-17 15:54:18,861] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_avg (avg=-0.00568798728528 std=0.950862037603) nulls=0
[2020-09-17 15:54:18,861] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (merchant_longitude)
[2020-09-17 15:54:18,862] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale merchant_longitude (avg=-99.9205120968 std=13.7225040612 shift=-99.9239807663 inv_scale=0.07335185565)
[2020-09-17 15:54:18,863] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled merchant_longitude (avg=0.000254433349916 std=1.00657113705) nulls=0
[2020-09-17 15:54:18,863] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_dow)
[2020-09-17 15:54:18,863] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_dow (avg=4.04510155317 std=1.91434463661 shift=4.03627740515 inv_scale=0.521398322544)
[2020-09-17 15:54:18,865] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_dow (avg=0.00460089597572 std=0.998136082297) nulls=0
[2020-09-17 15:54:18,865] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (w2_purchase_amount_max)
[2020-09-17 15:54:18,865] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale w2_purchase_amount_max (avg=1527.52985912 std=4157.25628983 shift=1550.57400591 inv_scale=0.000236875589795)
[2020-09-17 15:54:18,867] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled w2_purchase_amount_max (avg=-0.00545859586141 std=0.984752535581) nulls=0
[2020-09-17 15:54:18,867] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_amount)
[2020-09-17 15:54:18,868] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount (avg=251.596079749 std=714.801612144 shift=253.879229527 inv_scale=0.00147478881676)
[2020-09-17 15:54:18,869] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount (avg=-0.0033671637601 std=1.05418142379) nulls=0
[2020-09-17 15:54:18,869] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_longitude)
[2020-09-17 15:54:18,870] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_longitude (avg=-98.9778844086 std=13.6013718481 shift=-98.9478443162 inv_scale=0.0733747019088)
[2020-09-17 15:54:18,871] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_longitude (avg=-0.00220418282197 std=0.997996604903) nulls=0
[2020-09-17 15:54:18,871] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_amount_min)
[2020-09-17 15:54:18,872] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount_min (avg=85.269194544 std=331.352857937 shift=89.2651603013 inv_scale=0.00271090938669)
[2020-09-17 15:54:18,873] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount_min (avg=-0.0108327010804 std=0.898267572887) nulls=0
[2020-09-17 15:54:18,873] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_fico_score)
[2020-09-17 15:54:18,874] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_fico_score (avg=572.980734767 std=168.680499303 shift=572.291157774 inv_scale=0.00589611775131)
[2020-09-17 15:54:18,875] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_fico_score (avg=0.00406582715212 std=0.99456008624) nulls=0
[2020-09-17 15:54:18,875] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2020-09-17 15:54:18,876] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_location)
[2020-09-17 15:54:18,884] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 103) nnz=20088
[2020-09-17 15:54:18,885] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_state_enName)
[2020-09-17 15:54:18,891] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 48) nnz=20088
[2020-09-17 15:54:18,891] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (purchase_date)
[2020-09-17 15:54:18,898] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 103) nnz=20088
[2020-09-17 15:54:18,898] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_first_active_month)
[2020-09-17 15:54:18,904] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 74) nnz=20088
[2020-09-17 15:54:18,905] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_location)
[2020-09-17 15:54:18,911] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 103) nnz=20088
[2020-09-17 15:54:18,912] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_id)
[2020-09-17 15:54:18,918] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 103) nnz=20088
[2020-09-17 15:54:18,919] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (item_category)
[2020-09-17 15:54:18,924] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 6) nnz=20088
[2020-09-17 15:54:18,925] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_state)
[2020-09-17 15:54:18,931] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 48) nnz=20088
[2020-09-17 15:54:18,932] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state)
[2020-09-17 15:54:18,938] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 48) nnz=20088
[2020-09-17 15:54:18,938] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_id)
[2020-09-17 15:54:18,945] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 103) nnz=20088
[2020-09-17 15:54:18,945] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_subsector_description)
[2020-09-17 15:54:18,951] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 40) nnz=20088
[2020-09-17 15:54:18,951] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (card_reward_program)
[2020-09-17 15:54:18,957] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 5) nnz=20088
[2020-09-17 15:54:18,958] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state_enName)
[2020-09-17 15:54:18,964] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 48) nnz=20088
[2020-09-17 15:54:18,964] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,964] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(interaction)
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RealignTarget
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (20088,)
[2020-09-17 15:54:18,965] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (20088,)
[2020-09-17 15:54:18,966] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DropRowsWhereNoTarget
[2020-09-17 15:54:18,966] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because no target
[2020-09-17 15:54:18,966] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (20088, 855) target before = (20088,)
[2020-09-17 15:54:18,967] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(20088, 40)
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (20088, 855) target after = (20088,)
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (20088, 40) 
[2020-09-17 15:54:18,986] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(20088, 855) 
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((20088,))
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:EmitCurrentMFAsResult
[2020-09-17 15:54:18,987] [321164/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 20088
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (20088, 40) 
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((20088, 40))
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((20088, 855))
[2020-09-17 15:54:18,987] [321164/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((20088,))
[2020-09-17 15:54:18,987] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing test set
[2020-09-17 15:54:18,988] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Hyperparameter searching
[2020-09-17 15:54:18,988] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(79912,855) tn=68324760 nnz=2876832 fill_ratio=0.04
[2020-09-17 15:54:18,988] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Algorithm not supported, using NPA
[2020-09-17 15:54:19,846] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Create CLF from params: {u'computeLearningCurves': False, u'algorithm': u'RANDOM_FOREST_CLASSIFICATION', u'grid_search_params': {u'nIter': 0, u'nJobs': 4, u'nIterRandom': 5, u'bayesianOptimizer': u'SCIKIT_OPTIMIZE', u'strategy': u'GRID', u'randomized': True, u'shuffleIterations': 1, u'seed': 0, u'mode': u'KFOLD', u'timeout': 0, u'splitRatio': 0.8, u'nFolds': 3, u'stratified': True}, u'autoOptimizeThreshold': True, u'gridLength': 2, u'metrics': {u'customEvaluationMetricGIB': True, u'customEvaluationMetricNeedsProba': False, u'thresholdOptimizationMetric': u'F1', u'liftPoint': 0.4, u'costMatrixWeights': {u'fnGain': 0.0, u'tpGain': 1.0, u'tnGain': 0.0, u'fpGain': -0.3}, u'evaluationMetric': u'ROC_AUC'}, u'forcedClassifierThreshold': 0.0, u'rf_classifier_grid': {u'n_jobs': 4, u'max_tree_depth': {u'values': [6, 16], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'min': 1}, u'range': {u'scaling': u'LINEAR', u'max': 10, u'nbValues': 3, u'min': 5}}, u'selection_mode': u'auto', u'enabled': True, u'min_samples_leaf': {u'values': [1], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'min': 1}, u'range': {u'scaling': u'LINEAR', u'max': 20, u'nbValues': 3, u'min': 1}}, u'n_estimators': {u'values': [100], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'min': 1}, u'range': {u'scaling': u'LINEAR', u'max': 200, u'nbValues': 3, u'min': 80}}, u'max_feature_prop': {u'values': [0.3], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'max': 1.0, u'min': 1e-23}, u'range': {u'scaling': u'LINEAR', u'max': 0.7, u'nbValues': 3, u'min': 0.1}}, u'max_features': {u'values': [5], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'min': 1}, u'range': {u'scaling': u'LINEAR', u'max': 20, u'nbValues': 3, u'min': 1}}}, u'max_ensemble_nodes_serialized': 6000, u'pluginAlgoCustomGridSearch': False, u'skipExpensiveReports': False} for algorithm RANDOM_FOREST_CLASSIFICATION
[2020-09-17 15:54:19,847] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Using stratified K-Fold CV with k=3
[2020-09-17 15:54:19,870] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Fitting 3 folds for each of 2 candidates, totalling 6 fits
[2020-09-17 15:54:19,871] [321164/MainThread] [INFO] [dataiku.doctor.crossval.strategies.grid_search_strategy] Running GridSearchStrategy for hyperparameters space: <dataiku.doctor.prediction.common.TreesHyperparametersSpace object at 0x7f0264b4ba90>
[2020-09-17 15:54:19,871] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 6, 'min_samples_leaf': 1}
[2020-09-17 15:54:19,871] [321164/SS0-139648240412416] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Starting search slave 0
[2020-09-17 15:54:19,872] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 6, 'min_samples_leaf': 1}
[2020-09-17 15:54:19,874] [321164/SS0-139648240412416] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=0: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 
[2020-09-17 15:54:19,874] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Starting search slave 1
[2020-09-17 15:54:19,879] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 6, 'min_samples_leaf': 1}
[2020-09-17 15:54:19,880] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=1: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 
[2020-09-17 15:54:19,880] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Starting search slave 2
[2020-09-17 15:54:19,884] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 16, 'min_samples_leaf': 1}
[2020-09-17 15:54:19,889] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=2: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 
[2020-09-17 15:54:19,898] [321164/SS3-139648164759296] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Starting search slave 3
[2020-09-17 15:54:19,899] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 16, 'min_samples_leaf': 1}
[2020-09-17 15:54:19,900] [321164/SS3-139648164759296] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=0: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 
/data/home/dkuacademy/dss/installers/dataiku-dss-8.0.1/python.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2020-09-17 15:54:37,598] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=2: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 (ft=13.0s st=4.0s sc=0.744400113055, sg=1)
[2020-09-17 15:54:37,674] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=1: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 
[2020-09-17 15:54:37,705] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 16, 'min_samples_leaf': 1}
[2020-09-17 15:54:37,817] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=1: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 (ft=13.0s st=4.0s sc=0.732795477088, sg=1)
[2020-09-17 15:54:37,916] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Fit s=2: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 
[2020-09-17 15:54:38,303] [321164/SS0-139648240412416] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=0: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=6, min_samples_leaf=1 (ft=14.0s st=4.0s sc=0.735587521529, sg=1)
[2020-09-17 15:54:48,569] [321164/SS3-139648164759296] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=0: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 (ft=21.0s st=7.0s sc=0.756535525904, sg=1)
[2020-09-17 15:54:58,358] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=1: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 (ft=17.0s st=2.0s sc=0.752280681447, sg=1)
[2020-09-17 15:54:58,522] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.local_search_slave] Done s=2: max_features=auto, n_estimators=100, min_samples_split=3, max_depth=16, min_samples_leaf=1 (ft=17.0s st=2.0s sc=0.75969998311, sg=1)
[2020-09-17 15:54:59,019] [321164/Thread-1] [INFO] [dataiku.doctor.crossval.search_runner] Completed search for hyperparameters
[2020-09-17 15:54:59,020] [321164/Thread-1] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Slave pool has been soft interrupted
[2020-09-17 15:54:59,020] [321164/MainThread] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Slave pool has been hard interrupted (shutdown)
[2020-09-17 15:54:59,020] [321164/MainThread] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Stopping search slave 0
[2020-09-17 15:54:59,020] [321164/SS1-139648232019712] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Stopping search slave 1
[2020-09-17 15:54:59,020] [321164/SS2-139648220149504] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Stopping search slave 2
[2020-09-17 15:54:59,021] [321164/SS3-139648164759296] [INFO] [dataiku.doctor.crossval.distributed.search_slave_pool] Stopping search slave 3
[2020-09-17 15:54:59,021] [321164/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Hyperparameter search done, best_parameters being : {'max_features': u'auto', 'n_estimators': 100, 'min_samples_split': 3, 'max_depth': 16, 'min_samples_leaf': 1}
[2020-09-17 15:54:59,021] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Hyperparameter searching
[2020-09-17 15:54:59,058] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Fitting model
[2020-09-17 15:54:59,058] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Fitting model:
[2020-09-17 15:54:59,059] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common]   Model is: RandomForestClassifier(bootstrap=True,
            class_weight={0: 5.349578256794752, 1: 0.5515508744806261},
            criterion='gini', max_depth=16, max_features=u'auto',
            max_leaf_nodes=None, min_impurity_decrease=0.0,
            min_impurity_split=None, min_samples_leaf=1,
            min_samples_split=3, min_weight_fraction_leaf=0.0,
            n_estimators=100, n_jobs=4, oob_score=False, random_state=1337,
            verbose=0, warm_start=False)
[2020-09-17 15:54:59,059] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X class: <type 'numpy.ndarray'>
[2020-09-17 15:54:59,059] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X shape: (79912, 855)
[2020-09-17 15:54:59,059] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_y shape: (79912,)
[2020/09/17-15:55:07.886] [process-resource-monitor-321164-122061] [DEBUG] [dku.resource]  - Process stats for pid 321164: {"pid":321164,"commandName":"/data/home/dkuacademy/dss/data-dir-design-33333/bin/python","cpuUserTimeMS":214290,"cpuSystemTimeMS":5010,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":219300,"cpuCurrent":3.9692061587682463,"vmSizeMB":4975,"vmRSSMB":1153,"vmHWMMB":3616,"vmRSSAnonMB":1132,"vmDataMB":4305,"vmSizePeakMB":7710,"vmRSSPeakMB":3611,"vmRSSTotalMBS":130660,"majorFaults":146,"childrenMajorFaults":0}
[2020-09-17 15:55:11,450] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_fit] Obtained RF CLF params: {'warm_start': False, 'oob_score': False, 'n_jobs': 4, 'min_impurity_decrease': 0.0, 'verbose': 0, 'max_leaf_nodes': None, 'bootstrap': True, 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3, 'min_weight_fraction_leaf': 0.0, 'criterion': 'gini', 'random_state': 1337, 'min_impurity_split': None, 'max_features': u'auto', 'max_depth': 16, 'class_weight': {0: 5.349578256794752, 1: 0.5515508744806261}} 
[2020-09-17 15:55:11,450] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_fit] Output params are {'resolved': {'rf': {'max_tree_depth': 16, 'selection_mode': u'auto', 'min_samples_leaf': 1, 'estimators': 100}, 'skipExpensiveReports': False, 'algorithm': u'RANDOM_FOREST_CLASSIFICATION'}, 'other': {'rf_min_samples_split': 3}}
[2020-09-17 15:55:11,451] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Fitting model
[2020-09-17 15:55:11,451] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Saving model
[2020-09-17 15:55:11,494] [321164/MainThread] [INFO] [dataiku.doctor.prediction_entrypoints] PICKLING RandomForestClassifier(bootstrap=True,
            class_weight={0: 5.349578256794752, 1: 0.5515508744806261},
            criterion='gini', max_depth=16, max_features=u'auto',
            max_leaf_nodes=None, min_impurity_decrease=0.0,
            min_impurity_split=None, min_samples_leaf=1,
            min_samples_split=3, min_weight_fraction_leaf=0.0,
            n_estimators=100, n_jobs=4, oob_score=False, random_state=1337,
            verbose=0, warm_start=False)
[2020-09-17 15:55:11,542] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Saving model
[2020-09-17 15:55:11,542] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Scoring model
[2020-09-17 15:55:11,543] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Intrinsic scoring
[2020-09-17 15:55:11,842] [321164/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Computing feature importance
[2020-09-17 15:55:11,844] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Extracting rescalers
[2020-09-17 15:55:11,844] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Creating random forest trees summary
[2020-09-17 15:55:11,963] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(20088,855) tn=17175240 nnz=723168 fill_ratio=0.04
[2020-09-17 15:55:11,963] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] too small, using array
[2020-09-17 15:55:12,664] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Probas raw shape 20088/2 target_map=2
/data/home/dkuacademy/dss/installers/dataiku-dss-8.0.1/python.packages/sklearn/metrics/classification.py:543: RuntimeWarning: invalid value encountered in double_scalars
  mcc = cov_ytyp / np.sqrt(cov_ytyt * cov_ypyp)
/data/home/dkuacademy/dss/installers/dataiku-dss-8.0.1/python.packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 due to no predicted samples.
  'precision', 'predicted', average, warn_for)
[2020-09-17 15:55:15,121] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Starting threshold optim
[2020-09-17 15:55:15,124] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.000000 score 0.951456 (pred_true=20088)
[2020-09-17 15:55:15,127] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.025000 score 0.951456 (pred_true=20088)
[2020-09-17 15:55:15,129] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.050000 score 0.951456 (pred_true=20088)
[2020-09-17 15:55:15,132] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.075000 score 0.951456 (pred_true=20088)
[2020-09-17 15:55:15,135] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.100000 score 0.951481 (pred_true=20087)
[2020-09-17 15:55:15,137] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.125000 score 0.951556 (pred_true=20084)
[2020-09-17 15:55:15,140] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.150000 score 0.951730 (pred_true=20077)
[2020-09-17 15:55:15,143] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.175000 score 0.951898 (pred_true=20066)
[2020-09-17 15:55:15,146] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.200000 score 0.952269 (pred_true=20049)
[2020-09-17 15:55:15,148] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.225000 score 0.952376 (pred_true=20030)
[2020-09-17 15:55:15,151] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.250000 score 0.952540 (pred_true=19994)
[2020-09-17 15:55:15,154] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.275000 score 0.952812 (pred_true=19960)
[2020-09-17 15:55:15,157] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.300000 score 0.952586 (pred_true=19925)
[2020-09-17 15:55:15,159] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.325000 score 0.952573 (pred_true=19894)
[2020-09-17 15:55:15,162] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.350000 score 0.952208 (pred_true=19833)
[2020-09-17 15:55:15,165] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.375000 score 0.951408 (pred_true=19741)
[2020-09-17 15:55:15,168] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.400000 score 0.949472 (pred_true=19553)
[2020-09-17 15:55:15,170] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.425000 score 0.946573 (pred_true=19225)
[2020-09-17 15:55:15,173] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.450000 score 0.940022 (pred_true=18769)
[2020-09-17 15:55:15,176] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.475000 score 0.927061 (pred_true=18035)
[2020-09-17 15:55:15,179] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.500000 score 0.895250 (pred_true=16569)
[2020-09-17 15:55:15,182] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.525000 score 0.835624 (pred_true=14289)
[2020-09-17 15:55:15,186] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.550000 score 0.756228 (pred_true=11800)
[2020-09-17 15:55:15,189] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.575000 score 0.673526 (pred_true=9664)
[2020-09-17 15:55:15,192] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.600000 score 0.597584 (pred_true=8011)
[2020-09-17 15:55:15,195] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.625000 score 0.522083 (pred_true=6588)
[2020-09-17 15:55:15,198] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.650000 score 0.436766 (pred_true=5185)
[2020-09-17 15:55:15,201] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.675000 score 0.358997 (pred_true=4034)
[2020-09-17 15:55:15,204] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.700000 score 0.286050 (pred_true=3069)
[2020-09-17 15:55:15,207] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.725000 score 0.207056 (pred_true=2124)
[2020-09-17 15:55:15,210] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.750000 score 0.132316 (pred_true=1301)
[2020-09-17 15:55:15,212] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.775000 score 0.076493 (pred_true=728)
[2020-09-17 15:55:15,215] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.800000 score 0.032702 (pred_true=303)
[2020-09-17 15:55:15,218] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.825000 score 0.012756 (pred_true=117)
[2020-09-17 15:55:15,220] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.850000 score 0.005034 (pred_true=46)
[2020-09-17 15:55:15,223] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.875000 score 0.000877 (pred_true=8)
[2020-09-17 15:55:15,225] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.900000 score 0.000000 (pred_true=0)
[2020-09-17 15:55:15,228] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.925000 score 0.000000 (pred_true=0)
[2020-09-17 15:55:15,231] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.950000 score 0.000000 (pred_true=0)
[2020-09-17 15:55:15,233] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.975000 score 0.000000 (pred_true=0)
[2020-09-17 15:55:15,233] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Selected threshold 0.275 
[2020-09-17 15:55:15,235] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] preds 0        1
1        1
2        1
3        1
4        1
5        1
6        1
7        1
8        1
9        1
10       1
11       1
12       1
13       1
14       1
15       1
16       1
17       1
18       1
19       1
20       1
21       1
22       1
23       1
24       1
25       1
26       1
27       1
28       1
29       1
        ..
20058    1
20059    1
20060    1
20061    1
20062    1
20063    1
20064    1
20065    1
20066    1
20067    1
20068    1
20069    1
20070    1
20071    1
20072    1
20073    1
20074    1
20075    1
20076    1
20077    1
20078    1
20079    1
20080    1
20081    1
20082    1
20083    1
20084    1
20085    1
20086    1
20087    1
Name: predicted, Length: 20088, dtype: int64
[2020-09-17 15:55:15,235] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MAPPED SHAPE 20088
[2020-09-17 15:55:15,235] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] k=1 v=1
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Mask data [ True  True  True ...  True  True  True]
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] mapped pred <type 'numpy.ndarray'>
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] k=0 v=0
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Mask data [False False False ... False False False]
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] mapped pred <type 'numpy.ndarray'>
[2020-09-17 15:55:15,236] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MAPPED PREDS [u'1' u'1' u'1' ... u'1' u'1' u'1']
[2020-09-17 15:55:15,524] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Total true rate = 18228 / 20088
[2020-09-17 15:55:15,524] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Computing lift on first 8035 rows
[2020-09-17 15:55:15,524] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] True rate on considered : 8035 / 7863
[2020-09-17 15:55:15,524] [321164/MainThread] [INFO] [dataiku.doctor.prediction.common] Lift = 1.078450
[2020-09-17 15:55:15,554] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Density for 0 (id 0)
[2020-09-17 15:55:15,554] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] valid_y shape = (20088,)
[2020-09-17 15:55:15,554] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] CP: <type 'numpy.ndarray'> ((20088,))
[2020-09-17 15:55:15,554] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Actual shape = (20088,)
[2020-09-17 15:55:15,554] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MASK is <type 'numpy.ndarray'> 
[2020-09-17 15:55:15,555] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba [0.46081695 0.45277617 0.42881735 ... 0.52782641 0.46939309 0.5196017 ]
[2020-09-17 15:55:15,555] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba shape (1860,)
[2020-09-17 15:55:15,668] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Density for 1 (id 1)
[2020-09-17 15:55:15,668] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] valid_y shape = (20088,)
[2020-09-17 15:55:15,668] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] CP: <type 'numpy.ndarray'> ((20088,))
[2020-09-17 15:55:15,668] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Actual shape = (20088,)
[2020-09-17 15:55:15,668] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MASK is <type 'numpy.ndarray'> 
[2020-09-17 15:55:15,669] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba [0.54783929 0.5499766  0.56578045 ... 0.54657786 0.6599476  0.5113226 ]
[2020-09-17 15:55:15,669] [321164/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba shape (18228,)
/data/home/dkuacademy/dss/installers/dataiku-dss-8.0.1/python/dataiku/doctor/utils/calibration.py:34: RuntimeWarning: invalid value encountered in divide
  freq_pos = y_true[i_start:i_end].sum() / weight
/data/home/dkuacademy/dss/installers/dataiku-dss-8.0.1/python/dataiku/doctor/utils/calibration.py:35: RuntimeWarning: invalid value encountered in double_scalars
  avg_pred = y_prob[i_start:i_end].sum() / weight
[2020-09-17 15:55:17,012] [321164/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Scoring model
[2020-09-17 15:55:17,013] [321164/MainThread] [INFO] [root] Reading with dtypes: None
[2020-09-17 15:55:17,013] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:55:17,013] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <type 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_weekend: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month_parsed: None (schema_type=date feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,014] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_cardholder_distance: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_location: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_min: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_max: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for w2_purchase_amount_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2020-09-17 15:55:17,015] [321164/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'w2_purchase_amount_min': <type 'numpy.float64'>, u'signature_provided': <type 'numpy.float64'>, u'purchase_amount_max': <type 'numpy.float64'>, u'card_age': <type 'numpy.float64'>, u'card_latitude': <type 'numpy.float64'>, u'w2_purchase_amount_avg': <type 'numpy.float64'>, u'purchase_dow': <type 'numpy.float64'>, u'days_active': <type 'numpy.float64'>, u'merchant_location': 'str', u'purchase_date': 'str', u'merchant_latitude': <type 'numpy.float64'>, u'purchase_weekend': <type 'numpy.float64'>, u'purchase_month': <type 'numpy.float64'>, u'card_reward_program': 'str', u'card_state_enName': 'str', u'card_first_active_month': 'str', u'authorized_flag': <type 'numpy.object_'>, u'merchant_category_id': <type 'numpy.float64'>, u'merchant_cardholder_distance': <type 'numpy.float64'>, u'card_location': 'str', u'purchase_amount_avg': <type 'numpy.float64'>, u'merchant_longitude': <type 'numpy.float64'>, u'merchant_id': 'str', u'purchase_day': <type 'numpy.float64'>, u'w2_purchase_amount_max': <type 'numpy.float64'>, u'item_category': 'str', u'card_state': 'str', u'merchant_state': 'str', u'card_id': 'str', u'purchase_amount': <type 'numpy.float64'>, u'merchant_subsector_description': 'str', u'card_longitude': <type 'numpy.float64'>, u'purchase_amount_min': <type 'numpy.float64'>, u'merchant_state_enName': 'str', u'card_fico_score': <type 'numpy.float64'>}
[2020-09-17 15:55:17,212] [321164/MainThread] [INFO] [root] Loaded table
[2020-09-17 15:55:17,216] [321164/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Building background rows
[2020-09-17 15:55:17,218] [321164/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Background estimated bias <= 0.00667187577684
[2020-09-17 15:55:17,230] [321164/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Built background rows with shape=(1000, 40)
[2020-09-17 15:55:17,230] [321164/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Saving background rows with shape=(1000, 40)
[2020-09-17 15:55:17,263] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-01-01
1       2015-11-01
2       2017-10-01
3       2017-06-01
4       2016-04-01
5       2017-05-01
6       2016-12-01
7       2016-12-01
8       2017-11-01
9       2017-10-01
10      2016-02-01
11      2014-09-01
12      2014-08-01
13      2017-02-01
14      2016-12-01
15      2017-06-01
16      2017-07-01
17      2017-06-01
18      2017-06-01
19      2017-10-01
20      2017-10-01
21      2016-03-01
22      2017-05-01
23      2016-12-01
24      2017-07-01
25      2016-09-01
26      2017-10-01
27      2017-10-01
28      2016-06-01
29      2015-07-01
           ...    
20058   2017-06-01
20059   2015-04-01
20060   2017-01-01
20061   2017-10-01
20062   2016-07-01
20063   2017-01-01
20064   2017-06-01
20065   2017-07-01
20066   2017-04-01
20067   2017-04-01
20068   2016-08-01
20069   2017-02-01
20070   2017-01-01
20071   2016-02-01
20072   2017-06-01
20073   2017-02-01
20074   2016-11-01
20075   2017-03-01
20076   2016-06-01
20077   2016-06-01
20078   2017-02-01
20079   2017-03-01
20080   2017-06-01
20081   2016-11-01
20082   2017-02-01
20083   2017-01-01
20084   2016-11-01
20085   2016-10-01
20086   2017-05-01
20087   2017-06-01
Name: card_first_active_month_parsed, Length: 20088, dtype: datetime64[ns]
[2020-09-17 15:55:17,266] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.692218e+09
1        3.655325e+09
2        3.715805e+09
3        3.705264e+09
4        3.668458e+09
5        3.702586e+09
6        3.689539e+09
7        3.689539e+09
8        3.718483e+09
9        3.715805e+09
10       3.663274e+09
11       3.618518e+09
12       3.615840e+09
13       3.694896e+09
14       3.689539e+09
15       3.705264e+09
16       3.707856e+09
17       3.705264e+09
18       3.705264e+09
19       3.715805e+09
20       3.715805e+09
21       3.665779e+09
22       3.702586e+09
23       3.689539e+09
24       3.707856e+09
25       3.681677e+09
26       3.715805e+09
27       3.715805e+09
28       3.673728e+09
29       3.644698e+09
             ...     
20058    3.705264e+09
20059    3.636835e+09
20060    3.692218e+09
20061    3.715805e+09
20062    3.676320e+09
20063    3.692218e+09
20064    3.705264e+09
20065    3.707856e+09
20066    3.699994e+09
20067    3.699994e+09
20068    3.678998e+09
20069    3.694896e+09
20070    3.692218e+09
20071    3.663274e+09
20072    3.705264e+09
20073    3.694896e+09
20074    3.686947e+09
20075    3.697315e+09
20076    3.673728e+09
20077    3.673728e+09
20078    3.694896e+09
20079    3.697315e+09
20080    3.705264e+09
20081    3.686947e+09
20082    3.694896e+09
20083    3.692218e+09
20084    3.686947e+09
20085    3.684269e+09
20086    3.702586e+09
20087    3.705264e+09
Name: card_first_active_month_parsed, Length: 20088, dtype: float64
[2020-09-17 15:55:17,268] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-11-08
1       2017-11-08
2       2017-11-08
3       2017-11-08
4       2017-11-08
5       2017-11-08
6       2017-11-08
7       2017-11-08
8       2017-11-08
9       2017-11-08
10      2017-11-09
11      2017-11-09
12      2017-11-09
13      2017-11-09
14      2017-11-09
15      2017-11-09
16      2017-11-09
17      2017-11-09
18      2017-11-09
19      2017-11-09
20      2017-11-09
21      2017-11-09
22      2017-11-10
23      2017-11-10
24      2017-11-10
25      2017-11-10
26      2017-11-10
27      2017-11-10
28      2017-11-10
29      2017-11-10
           ...    
20058   2017-07-14
20059   2017-08-10
20060   2017-08-11
20061   2017-10-18
20062   2017-10-28
20063   2017-11-22
20064   2017-11-24
20065   2017-12-20
20066   2017-08-06
20067   2017-04-26
20068   2017-06-09
20069   2017-07-07
20070   2017-01-31
20071   2017-02-17
20072   2017-09-19
20073   2017-12-08
20074   2017-03-17
20075   2017-04-04
20076   2017-05-03
20077   2017-05-07
20078   2017-06-10
20079   2017-06-22
20080   2017-06-28
20081   2017-07-24
20082   2017-07-29
20083   2017-08-01
20084   2017-08-02
20085   2017-08-07
20086   2017-08-15
20087   2017-08-16
Name: purchase_date_parsed, Length: 20088, dtype: datetime64[ns]
[2020-09-17 15:55:17,272] [321164/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.719088e+09
1        3.719088e+09
2        3.719088e+09
3        3.719088e+09
4        3.719088e+09
5        3.719088e+09
6        3.719088e+09
7        3.719088e+09
8        3.719088e+09
9        3.719088e+09
10       3.719174e+09
11       3.719174e+09
12       3.719174e+09
13       3.719174e+09
14       3.719174e+09
15       3.719174e+09
16       3.719174e+09
17       3.719174e+09
18       3.719174e+09
19       3.719174e+09
20       3.719174e+09
21       3.719174e+09
22       3.719261e+09
23       3.719261e+09
24       3.719261e+09
25       3.719261e+09
26       3.719261e+09
27       3.719261e+09
28       3.719261e+09
29       3.719261e+09
             ...     
20058    3.708979e+09
20059    3.711312e+09
20060    3.711398e+09
20061    3.717274e+09
20062    3.718138e+09
20063    3.720298e+09
20064    3.720470e+09
20065    3.722717e+09
20066    3.710966e+09
20067    3.702154e+09
20068    3.705955e+09
20069    3.708374e+09
20070    3.694810e+09
20071    3.696278e+09
20072    3.714768e+09
20073    3.721680e+09
20074    3.698698e+09
20075    3.700253e+09
20076    3.702758e+09
20077    3.703104e+09
20078    3.706042e+09
20079    3.707078e+09
20080    3.707597e+09
20081    3.709843e+09
20082    3.710275e+09
20083    3.710534e+09
20084    3.710621e+09
20085    3.711053e+09
20086    3.711744e+09
20087    3.711830e+09
Name: purchase_date_parsed, Length: 20088, dtype: float64
[2020-09-17 15:55:17,273] [321164/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2020-09-17 15:55:17,277] [321164/MainThread] [INFO] [dataiku.doctor.prediction.histogram_handler] Computing histograms
[2020-09-17 15:55:31,932] [321164/MainThread] [INFO] [dataiku.doctor.prediction.histogram_handler] Saving histograms
[2020-09-17 15:55:32,062] [321164/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Computing column importance
[2020-09-17 15:55:32,063] [321164/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Reusing the model feature importance
[2020-09-17 15:55:32,065] [321164/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Saving column importance
[2020/09/17-15:55:32.177] [MRT-122051] [INFO] [dku.block.link.interaction]  - Check result for nullity exceptionIfNull=true result=not null
[2020/09/17-15:55:32.177] [MRT-122051] [INFO] [dku.analysis.prediction]  - Training returned ok
[2020/09/17-15:55:32.178] [MRT-122051] [INFO] [dku.ml.versioning]  - Dumping version info {"trainedWithDSSVersion":"8.0.1","trainedWithDSSConfVersion":"8000","trainedWithDSSBackend":"PY_MEMORY","backendCompatibilityVersion":18} in /data/home/dkuacademy/dss/data-dir-design-33333/analysis-data/CREDITCARDFRAUDSTART3/LvXsfpWd/MbtGxk2T/sessions/s1/pp1/m1
[2020/09/17-15:55:32.219] [KNL-python-single-command-kernel-out-122062] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2020/09/17-15:55:32.220] [KNL-python-single-command-kernel-monitor-122059] [INFO] [dku.kernels]  - Process done with code 143
[2020/09/17-15:55:32.219] [KNL-python-single-command-kernel-err-122063] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2020/09/17-15:55:32.222] [KNL-python-single-command-kernel-monitor-122059] [WARN] [dku.resource]  - stat file for pid 321164 does not exist. Process died?
[2020/09/17-15:55:32.222] [KNL-python-single-command-kernel-monitor-122059] [INFO] [dku.resourceusage]  - Reporting completion of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"christelle","projectKey":"CREDITCARDFRAUDSTART3","analysisId":"LvXsfpWd","mlTaskId":"MbtGxk2T","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"vqW6ZLpdt4q4M4b9","startTime":1600350847863,"localProcess":{"pid":321164,"commandName":"/data/home/dkuacademy/dss/data-dir-design-33333/bin/python","cpuUserTimeMS":244010,"cpuSystemTimeMS":5270,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":249280,"cpuCurrent":0.9978004399120176,"vmSizeMB":4755,"vmRSSMB":949,"vmHWMMB":3616,"vmRSSAnonMB":927,"vmDataMB":4085,"vmSizePeakMB":7710,"vmRSSPeakMB":3611,"vmRSSTotalMBS":149370,"majorFaults":147,"childrenMajorFaults":0}}
[2020/09/17-15:55:32.222] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.ml.python] T-MbtGxk2T - Processing thread joined ...
[2020/09/17-15:55:32.223] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.ml.python] T-MbtGxk2T - Joining processing thread ...
[2020/09/17-15:55:39.110] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.ml.python] T-MbtGxk2T - Processing thread joined ...
[2020/09/17-15:55:39.111] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction] T-MbtGxk2T - Train done
[2020/09/17-15:55:39.111] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction] T-MbtGxk2T - Train done
[2020/09/17-15:55:39.116] [FT-TrainWorkThread-pTDZ0A63-122050] [INFO] [dku.analysis.prediction] T-MbtGxk2T - Publishing mltask-train-done reflected event
