[2022/11/16-15:58:53.191] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.prediction]  - ******************************************
[2022/11/16-15:58:53.192] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.prediction]  - ** Start train session s1
[2022/11/16-15:58:53.192] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.prediction]  - ******************************************
[2022/11/16-15:58:53.194] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.data] T-f8fZYP6m - [ct: 3] Need to compute sampleId before checking memory cache
[2022/11/16-15:58:53.195] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dip.shaker.runner] T-f8fZYP6m - [ct: 4] Script settings sampleMax=104857600 processedMax=-1
[2022/11/16-15:58:53.195] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dip.shaker.runner] T-f8fZYP6m - [ct: 4] Processing with sampleMax=104857600 processedMax=524288000
[2022/11/16-15:58:53.196] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dip.shaker.runner] T-f8fZYP6m - [ct: 5] Computed required sample id : 610920c20be262f999102e9f9409e670-NA-ac2e0aa81c1215a34ba9f85052ba5ff70--d751713988987e9331980363e24189ce
[2022/11/16-15:58:53.196] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.shaker.cache] T-f8fZYP6m - Shaker MemoryCache get on dataset MLOPS_V11_STARTER_TEST.train key=ds=0aec1cdbc2d5d96b5eef59f29a6e7790--scr=41f1140dbc049029fe6b7780036a7250--samp=610920c20be262f999102e9f9409e670-NA-ac2e0aa81c1215a34ba9f85052ba5ff70--d751713988987e9331980363e24189ce: hit
[2022/11/16-15:58:53.197] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 6] Column transaction_id meaning=LongMeaning fail=0
[2022/11/16-15:58:53.197] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 6] Column authorized_flag meaning=LongMeaning fail=0
[2022/11/16-15:58:53.197] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 6] Column purchase_date meaning=DateSource fail=0
[2022/11/16-15:58:53.197] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 6] Column purchase_date_parsed meaning=Date fail=0
[2022/11/16-15:58:53.198] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 7] Column purchase_year meaning=LongMeaning fail=0
[2022/11/16-15:58:53.198] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 7] Column purchase_month meaning=LongMeaning fail=0
[2022/11/16-15:58:53.198] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 7] Column purchase_day meaning=LongMeaning fail=0
[2022/11/16-15:58:53.198] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 7] Column purchase_dow meaning=LongMeaning fail=0
[2022/11/16-15:58:53.199] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 8] Column purchase_hour meaning=LongMeaning fail=0
[2022/11/16-15:58:53.199] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 8] Column date meaning=DateSource fail=0
[2022/11/16-15:58:53.199] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 8] Column card_id meaning=Text fail=0
[2022/11/16-15:58:53.199] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 8] Column merchant_id meaning=Text fail=0
[2022/11/16-15:58:53.200] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 9] Column merchant_category_id meaning=LongMeaning fail=0
[2022/11/16-15:58:53.200] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 9] Column item_category meaning=Text fail=0
[2022/11/16-15:58:53.200] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 9] Column purchase_amount meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.200] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 9] Column signature_provided meaning=LongMeaning fail=0
[2022/11/16-15:58:53.200] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 9] Column merchant_subsector_description meaning=Text fail=0
[2022/11/16-15:58:53.201] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 10] Column spending_type meaning=Text fail=0
[2022/11/16-15:58:53.201] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 10] Column merchant_latitude meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.201] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 10] Column merchant_longitude meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.201] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 10] Column card_first_active_month meaning=Date fail=0
[2022/11/16-15:58:53.202] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 11] Column days_active meaning=LongMeaning fail=0
[2022/11/16-15:58:53.202] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 11] Column card_reward_program meaning=Text fail=0
[2022/11/16-15:58:53.202] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 11] Column card_latitude meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.202] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 11] Column card_longitude meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.202] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 11] Column card_fico_score meaning=LongMeaning fail=0
[2022/11/16-15:58:53.203] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 12] Column card_age meaning=LongMeaning fail=0
[2022/11/16-15:58:53.203] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 12] Column card_geopoint meaning=GeoPoint fail=0
[2022/11/16-15:58:53.203] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 12] Column card_state meaning=USStateMeaning fail=0
[2022/11/16-15:58:53.203] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 12] Column card_state_enName meaning=USStateMeaning fail=0
[2022/11/16-15:58:53.204] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 13] Column merchant_geopoint meaning=GeoPoint fail=0
[2022/11/16-15:58:53.204] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 13] Column merchant_state meaning=USStateMeaning fail=0
[2022/11/16-15:58:53.204] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 13] Column merchant_state_enName meaning=USStateMeaning fail=0
[2022/11/16-15:58:53.204] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 13] Column dist_cardholder_merchant meaning=DoubleMeaning fail=0
[2022/11/16-15:58:53.204] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.shaker.schema] T-f8fZYP6m - [ct: 13] Column risk meaning=Text fail=0
[2022/11/16-15:58:53.205] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.file] T-f8fZYP6m - [ct: 14] Building Filesystem handler config: {"connection":"filesystem_managed","path":"MLOPS_V11_STARTER_TEST/train","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}}
[2022/11/16-15:58:53.206] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.ftplike] T-f8fZYP6m - Enumerating Filesystem dataset prefix=
[2022/11/16-15:58:53.206] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.datasets.fsbased] T-f8fZYP6m - [ct: 15] Building FS provider for dataset handler: MLOPS_V11_STARTER_TEST.train
[2022/11/16-15:58:53.206] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.datasets.fsbased] T-f8fZYP6m - [ct: 15] FS Provider built
[2022/11/16-15:58:53.207] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.fs.local] T-f8fZYP6m - [ct: 16] Enumerating local filesystem prefix=/
[2022/11/16-15:58:53.207] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.fs.local] T-f8fZYP6m - [ct: 16] Enumeration done nb_paths=1 size=2846170
[2022/11/16-15:58:53.207] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.input.push] T-f8fZYP6m - USTP: push selection.method=CLASS_REBALANCE_TARGET_NB_APPROX records=100000 ratio=0.02 col=authorized_flag
[2022/11/16-15:58:53.207] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.file] T-f8fZYP6m - [ct: 16] Building Filesystem handler config: {"connection":"filesystem_managed","path":"MLOPS_V11_STARTER_TEST/train","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}}
[2022/11/16-15:58:53.208] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.push] T-f8fZYP6m - [ct: 17] Counting values using full-push-with-filter
[2022/11/16-15:58:53.208] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.file] T-f8fZYP6m - [ct: 17] Building Filesystem handler config: {"connection":"filesystem_managed","path":"MLOPS_V11_STARTER_TEST/train","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}}
[2022/11/16-15:58:53.208] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.datasets.ftplike] T-f8fZYP6m - Enumerating Filesystem dataset prefix=
[2022/11/16-15:58:53.208] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.datasets.fsbased] T-f8fZYP6m - [ct: 17] Building FS provider for dataset handler: MLOPS_V11_STARTER_TEST.train
[2022/11/16-15:58:53.209] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.datasets.fsbased] T-f8fZYP6m - [ct: 18] FS Provider built
[2022/11/16-15:58:53.209] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.fs.local] T-f8fZYP6m - [ct: 18] Enumerating local filesystem prefix=/
[2022/11/16-15:58:53.209] [FT-TrainWorkThread-zIAftYjN-246982] [DEBUG] [dku.fs.local] T-f8fZYP6m - [ct: 18] Enumeration done nb_paths=1 size=2846170
[2022/11/16-15:58:53.210] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.input.push] T-f8fZYP6m - USTP: push selection.method=FULL records=-1 ratio=0.02 col=null
[2022/11/16-15:58:53.210] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 19] Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0
[2022/11/16-15:58:53.210] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.210] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.211] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 20] Start compressed [GZIP] stream: /data/home/dkuacademy/Library/dss-11.0/design/managed_datasets/MLOPS_V11_STARTER_TEST/train/out-s0.csv.gz / totalRecsBefore=0
[2022/11/16-15:58:53.211] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.211] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.570] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 379] after stream totalComp=2846170 totalUncomp=12771786 totalRec=39656
[2022/11/16-15:58:53.570] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 379] Extractor run done, totalCompressed=2846170 totalRecords=39656
[2022/11/16-15:58:53.570] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.sampling.rebalance] T-f8fZYP6m - [ct: 379] Column rebalance targetRecords=100000.000000 targetCountPerModality=50000.000000 missing=60344 largeClasses=0
[2022/11/16-15:58:53.573] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.sampling.rebalance] T-f8fZYP6m - [ct: 382] Column rebalance fixed targetCountPerModality=50000.000000d
[2022/11/16-15:58:53.573] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.sampling.rebalance] T-f8fZYP6m - [ct: 382]  value=0 totalCount=4061 proba=12.312238
[2022/11/16-15:58:53.574] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.sampling.rebalance] T-f8fZYP6m - [ct: 383]  value=1 totalCount=35595 proba=1.404692
[2022/11/16-15:58:53.574] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 383] Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0
[2022/11/16-15:58:53.575] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.575] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.576] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 385] Start compressed [GZIP] stream: /data/home/dkuacademy/Library/dss-11.0/design/managed_datasets/MLOPS_V11_STARTER_TEST/train/out-s0.csv.gz / totalRecsBefore=0
[2022/11/16-15:58:53.576] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:53.576] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku] T-f8fZYP6m - getCompression filename=**out-s0.csv.gz**
[2022/11/16-15:58:54.757] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 1566] after stream totalComp=2846170 totalUncomp=12771786 totalRec=39656
[2022/11/16-15:58:54.757] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.format] T-f8fZYP6m - [ct: 1566] Extractor run done, totalCompressed=2846170 totalRecords=39656
[2022/11/16-15:58:54.760] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1569] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337, instance id: db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.760] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1569] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337 i=db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.761] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1570] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337 i=db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.762] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1571] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337, instance id: db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.762] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1571] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337 i=db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.763] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.splits] T-f8fZYP6m - [ct: 1572] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=train,sel=(method=class-rebalance-fixednb-approx,column=authorized_flag,records=100000),r=0.8,s=1337 i=db1f475b1f2b7fea3077cfc2a3d90e18-0
[2022/11/16-15:58:54.765] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.ml.python] T-f8fZYP6m - [ct: 1574] Joining processing thread ...
[2022/11/16-15:58:54.765] [MRT-246983] [INFO] [dku.analysis.ml.python]  - Running a preprocessing set: pp1 in /data/home/dkuacademy/Library/dss-11.0/design/analysis-data/MLOPS_V11_STARTER_TEST/GvRJLWvO/f8fZYP6m/sessions/s1/pp1
[2022/11/16-15:58:54.766] [MRT-246983] [INFO] [dku.block.link]  - Started a socket on port 42503
[2022/11/16-15:58:54.767] [MRT-246983] [INFO] [dku.ml.kernel]  - Writing output of python-single-command-kernel to /data/home/dkuacademy/Library/dss-11.0/design/analysis-data/MLOPS_V11_STARTER_TEST/GvRJLWvO/f8fZYP6m/sessions/s1/pp1/train.log
[2022/11/16-15:58:54.767] [MRT-246983] [INFO] [dku.code.envs.resolution]  - Executing Python activity in builtin env
[2022/11/16-15:58:54.767] [MRT-246983] [INFO] [dku.code.projectLibs]  - EXTERNAL LIBS FROM MLOPS_V11_STARTER_TEST is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]}
[2022/11/16-15:58:54.767] [MRT-246983] [INFO] [dku.code.projectLibs]  - chunkFolder is /data/home/dkuacademy/Library/dss-11.0/design/config/projects/MLOPS_V11_STARTER_TEST/lib/R
[2022/11/16-15:58:54.767] [MRT-246983] [INFO] [dku.python.single_command.kernel]  - Starting Python process for kernel  python-single-command-kernel
[2022/11/16-15:58:54.768] [MRT-246983] [INFO] [dip.tickets]  - Creating API ticket for analysis-ml-MLOPS_V11_STARTER_TEST-ee92CyB on behalf of sean id=analysis-ml-MLOPS_V11_STARTER_TEST-ee92CyB_fuHdtSSunAEy
[2022/11/16-15:58:54.768] [MRT-246983] [INFO] [dku.security.process]  - Starting process (regular)
[2022/11/16-15:58:54.772] [MRT-246983] [INFO] [dku.security.process]  - Process started with pid=2071026
[2022/11/16-15:58:54.772] [MRT-246983] [INFO] [dku.processes.cgroups]  - Will use cgroups []
[2022/11/16-15:58:54.772] [MRT-246983] [INFO] [dku.processes.cgroups]  - Applying rules to used cgroups: []
[2022/11/16-15:58:54.778] [KNL-python-single-command-kernel-monitor-246992] [DEBUG] [dku.resourceusage]  - Reporting start of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"sean","projectKey":"MLOPS_V11_STARTER_TEST","analysisId":"GvRJLWvO","mlTaskId":"f8fZYP6m","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"7hOjVN0SpCuZLuT9","startTime":1668614334778,"localProcess":{"cpuCurrent":0.0}}
[2022/11/16-15:58:54.783] [process-resource-monitor-2071026-246996] [DEBUG] [dku.resource]  - Process stats for pid 2071026: {"pid":2071026,"commandName":"/data/home/dkuacademy/Library/dss-11.0/design/bin/python","cpuUserTimeMS":0,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":0,"cpuCurrent":0.0,"vmSizeMB":236,"vmRSSMB":8,"vmHWMMB":8,"vmRSSAnonMB":2,"vmDataMB":3,"vmSizePeakMB":236,"vmRSSPeakMB":8,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}
Installing debugging signal handler
[2022-11-16 15:58:55,496] [2071026/MainThread] [INFO] [dataiku.base.socket_block_link] Connecting to localhost (127.0.0.1) at port 42503
[2022-11-16 15:58:55,496] [2071026/MainThread] [INFO] [dataiku.base.socket_block_link] Connected to localhost (127.0.0.1) at port 42503
[2022/11/16-15:58:55.500] [MRT-246983] [INFO] [dku.link.secret_protected]  - Connected to kernel
[2022/11/16-15:58:55.502] [MRT-246983] [INFO] [dku.block.link.interaction]  - Execute link command respClazz=true respTypeToken=false respIsString=false is=false asyncInputStream=false os=false
[2022-11-16 15:58:55,555] [2071026/MainThread] [INFO] [dataiku.doctor.utils.dku_pickle] Setting cloudpickle as the pickling tool
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/dkuapi.py:16: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(api)
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [root] Running analysis command: train_prediction_models_nosave
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: DatasetSanityCheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_DATASET_SANITY_CHECKS
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: ClassifierAccuracyCheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_MODEL_CHECK
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: RegressionR2CheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_MODEL_CHECK
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: LeakageDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_LEAKAGE_DETECTION
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: TreeOverfitDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_TRAINING_OVERFIT
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: MLAssertionsDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_ML_ASSERTIONS
[2022-11-16 15:58:55,613] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: ModelCheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_MODEL_CHECK
[2022-11-16 15:58:55,614] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: AbnormalPredictionsDetectionDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_ABNORMAL_PREDICTIONS_DETECTION
[2022-11-16 15:58:55,614] [2071026/MainThread] [INFO] [dataiku.doctor.commands] PPS is {'feature_selection_params': {'method': 'NONE', 'random_forest_params': {'n_trees': 30, 'depth': 10, 'n_features': 25}, 'lasso_params': {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], 'cross_validate': True}, 'pca_params': {'n_features': 25, 'variance_proportion': 0.9}, 'correlation_params': {'min_abs_correlation': 0.0, 'max_abs_correlation': 1.0, 'n_features': 25}, 'custom_params': {'code': '# type your code here'}}, 'preprocessingFitSampleRatio': 1.0, 'preprocessingFitSampleSeed': 1337, 'target_remapping': [{'sourceValue': '0', 'mappedValue': 0, 'sampleFreq': 944}, {'sourceValue': '1', 'mappedValue': 1, 'sampleFreq': 9056}], 'skipPreprocessing': False, 'per_feature': {'date': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DateSource'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_latitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'spending_type': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_day': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_month': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_fico_score': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_id': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_longitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'item_category': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'dist_cardholder_merchant': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_state_enName': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_date_parsed': {'generate_derivative': False, 'numerical_handling': 'DATETIME_CYCLICAL', 'missing_handling': 'DROP_ROW', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': ['DAY', 'HOUR', 'MINUTE', 'WEEK'], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Date'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_state': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_state_enName': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'signature_provided': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_dow': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_age': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'transaction_id': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_IDENTIFIER', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_year': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_first_active_month': {'generate_derivative': False, 'numerical_handling': 'DATETIME_CYCLICAL', 'missing_handling': 'DROP_ROW', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': ['QUARTER', 'WEEK', 'YEAR'], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Date'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'days_active': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_latitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_category_id': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_id': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_subsector_description': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_longitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_hour': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_geopoint': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'GeoPoint'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_amount': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_reward_program': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_state': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_geopoint': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'GeoPoint'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'risk': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_date': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DateSource'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'authorized_flag': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'TARGET', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}}, 'reduce': {'enabled': False, 'kept_variance': 0.0}, 'feature_generation': {'pairwise_linear': {'behavior': 'DISABLED'}, 'polynomial_combinations': {'behavior': 'DISABLED'}, 'manual_interactions': {'interactions': []}, 'numericals_clustering': {'k': 0, 'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}, 'categoricals_count_transformer': {'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}}}
[2022-11-16 15:58:55,614] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading train set
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [root] Reading with dtypes: None
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <class 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <class 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for spending_type: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,615] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,616] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:58:55,616] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,616] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dist_cardholder_merchant: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,616] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for risk: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,616] [2071026/MainThread] [INFO] [root] Reading with FIXED dtypes: {'authorized_flag': <class 'numpy.object_'>, 'purchase_amount': <class 'numpy.float64'>, 'signature_provided': <class 'numpy.float64'>, 'merchant_subsector_description': 'str', 'card_fico_score': <class 'numpy.float64'>, 'card_age': <class 'numpy.float64'>, 'merchant_state': 'str'}
[2022-11-16 15:58:55,900] [2071026/MainThread] [INFO] [root] Loaded table
[2022-11-16 15:58:55,902] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-01-01 00:00:59
1       2017-01-01 00:21:40
2       2017-01-01 00:58:34
3       2017-01-01 01:09:52
4       2017-01-01 01:26:59
5       2017-01-01 01:34:19
6       2017-01-01 01:35:42
7       2017-01-01 01:57:45
8       2017-01-01 02:31:18
9       2017-01-01 03:26:33
10      2017-01-01 03:32:02
11      2017-01-01 03:50:52
12      2017-01-01 05:10:03
13      2017-01-01 05:35:05
14      2017-01-01 07:22:04
15      2017-01-01 08:00:13
16      2017-01-01 08:48:45
17      2017-01-01 09:01:54
18      2017-01-01 09:12:58
19      2017-01-01 09:28:44
20      2017-01-01 09:33:51
21      2017-01-01 09:38:16
22      2017-01-01 09:47:15
23      2017-01-01 10:01:45
24      2017-01-01 10:21:58
25      2017-01-01 10:35:16
26      2017-01-01 10:35:46
27      2017-01-01 10:36:27
28      2017-01-01 10:49:58
29      2017-01-01 10:50:30
                ...        
31713   2017-03-31 23:17:09
31714   2017-03-31 23:18:22
31715   2017-03-31 23:18:26
31716   2017-03-31 23:20:19
31717   2017-03-31 23:25:24
31718   2017-03-31 23:26:43
31719   2017-03-31 23:30:41
31720   2017-03-31 23:32:25
31721   2017-03-31 23:34:33
31722   2017-03-31 23:34:45
31723   2017-03-31 23:39:16
31724   2017-03-31 23:40:24
31725   2017-03-31 23:41:09
31726   2017-03-31 23:41:30
31727   2017-03-31 23:42:24
31728   2017-03-31 23:43:18
31729   2017-03-31 23:44:03
31730   2017-03-31 23:44:45
31731   2017-03-31 23:45:23
31732   2017-03-31 23:47:45
31733   2017-03-31 23:47:53
31734   2017-03-31 23:48:58
31735   2017-03-31 23:49:00
31736   2017-03-31 23:49:35
31737   2017-03-31 23:52:13
31738   2017-03-31 23:55:05
31739   2017-03-31 23:55:35
31740   2017-03-31 23:55:37
31741   2017-03-31 23:56:53
31742   2017-03-31 23:58:07
Name: purchase_date_parsed, Length: 31743, dtype: datetime64[ns]
[2022-11-16 15:58:55,905] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.692218e+09
1        3.692219e+09
2        3.692221e+09
3        3.692222e+09
4        3.692223e+09
5        3.692223e+09
6        3.692223e+09
7        3.692225e+09
8        3.692227e+09
9        3.692230e+09
10       3.692230e+09
11       3.692231e+09
12       3.692236e+09
13       3.692238e+09
14       3.692244e+09
15       3.692246e+09
16       3.692249e+09
17       3.692250e+09
18       3.692251e+09
19       3.692252e+09
20       3.692252e+09
21       3.692252e+09
22       3.692253e+09
23       3.692254e+09
24       3.692255e+09
25       3.692256e+09
26       3.692256e+09
27       3.692256e+09
28       3.692257e+09
29       3.692257e+09
             ...     
31713    3.699991e+09
31714    3.699991e+09
31715    3.699991e+09
31716    3.699991e+09
31717    3.699992e+09
31718    3.699992e+09
31719    3.699992e+09
31720    3.699992e+09
31721    3.699992e+09
31722    3.699992e+09
31723    3.699992e+09
31724    3.699992e+09
31725    3.699992e+09
31726    3.699992e+09
31727    3.699993e+09
31728    3.699993e+09
31729    3.699993e+09
31730    3.699993e+09
31731    3.699993e+09
31732    3.699993e+09
31733    3.699993e+09
31734    3.699993e+09
31735    3.699993e+09
31736    3.699993e+09
31737    3.699993e+09
31738    3.699993e+09
31739    3.699993e+09
31740    3.699993e+09
31741    3.699993e+09
31742    3.699993e+09
Name: purchase_date_parsed, Length: 31743, dtype: float64
[2022-11-16 15:58:55,907] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2020-12-01
1       2019-12-01
2       2019-05-01
3       2020-09-01
4       2019-10-01
5       2020-05-01
6       2020-07-01
7       2020-11-01
8       2020-07-01
9       2020-06-01
10      2020-05-01
11      2019-11-01
12      2019-02-01
13      2019-08-01
14      2020-03-01
15      2018-10-01
16      2020-10-01
17      2020-10-01
18      2020-11-01
19      2020-09-01
20      2020-08-01
21      2020-08-01
22      2020-12-01
23      2020-02-01
24      2019-07-01
25      2019-11-01
26      2020-10-01
27      2020-12-01
28      2020-11-01
29      2020-10-01
           ...    
31713   2020-12-01
31714   2018-05-01
31715   2020-12-01
31716   2020-06-01
31717   2021-03-01
31718   2021-03-01
31719   2021-03-01
31720   2021-07-01
31721   2021-02-01
31722   2019-11-01
31723   2020-10-01
31724   2020-02-01
31725   2021-01-01
31726   2020-06-01
31727   2020-05-01
31728   2019-05-01
31729   2021-03-01
31730   2021-01-01
31731   2020-02-01
31732   2020-11-01
31733   2021-03-01
31734   2020-12-01
31735   2020-03-01
31736   2020-09-01
31737   2021-03-01
31738   2020-11-01
31739   2021-01-01
31740   2021-02-01
31741   2020-07-01
31742   2021-02-01
Name: card_first_active_month, Length: 31743, dtype: datetime64[ns]
[2022-11-16 15:58:55,910] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.815770e+09
1        3.784147e+09
2        3.765658e+09
3        3.807907e+09
4        3.778877e+09
5        3.797280e+09
6        3.802550e+09
7        3.813178e+09
8        3.802550e+09
9        3.799958e+09
10       3.797280e+09
11       3.781555e+09
12       3.757968e+09
13       3.773606e+09
14       3.792010e+09
15       3.747341e+09
16       3.810499e+09
17       3.810499e+09
18       3.813178e+09
19       3.807907e+09
20       3.805229e+09
21       3.805229e+09
22       3.815770e+09
23       3.789504e+09
24       3.770928e+09
25       3.781555e+09
26       3.810499e+09
27       3.815770e+09
28       3.813178e+09
29       3.810499e+09
             ...     
31713    3.815770e+09
31714    3.734122e+09
31715    3.815770e+09
31716    3.799958e+09
31717    3.823546e+09
31718    3.823546e+09
31719    3.823546e+09
31720    3.834086e+09
31721    3.821126e+09
31722    3.781555e+09
31723    3.810499e+09
31724    3.789504e+09
31725    3.818448e+09
31726    3.799958e+09
31727    3.797280e+09
31728    3.765658e+09
31729    3.823546e+09
31730    3.818448e+09
31731    3.789504e+09
31732    3.813178e+09
31733    3.823546e+09
31734    3.815770e+09
31735    3.792010e+09
31736    3.807907e+09
31737    3.823546e+09
31738    3.813178e+09
31739    3.818448e+09
31740    3.821126e+09
31741    3.802550e+09
31742    3.821126e+09
Name: card_first_active_month, Length: 31743, dtype: float64
[2022-11-16 15:58:55,910] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Coercion done
[2022-11-16 15:58:55,913] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] Adding diagnostic: <DoctorDiagnostic ML_DIAGNOSTICS_DATASET_SANITY_CHECKS: The train dataset is imbalanced (balance=0.47), metrics can be misleading>
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded train df: shape=(31743,35)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : transaction_id (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : authorized_flag (object)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_date (object)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_date_parsed (float64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_year (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_month (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_day (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_dow (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_hour (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : date (object)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_id (object)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_id (object)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_category_id (int64)
[2022-11-16 15:58:55,914] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : item_category (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : purchase_amount (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : signature_provided (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_subsector_description (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : spending_type (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_latitude (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_longitude (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_first_active_month (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : days_active (int64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_reward_program (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_latitude (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_longitude (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_fico_score (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_age (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_geopoint (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_state (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : card_state_enName (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_geopoint (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_state (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : merchant_state_enName (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : dist_cardholder_merchant (float64)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.commands] Train col : risk (object)
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading train set
[2022-11-16 15:58:55,915] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading test set
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [root] Reading with dtypes: None
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <class 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <class 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for spending_type: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,916] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dist_cardholder_merchant: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for risk: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:58:55,917] [2071026/MainThread] [INFO] [root] Reading with FIXED dtypes: {'authorized_flag': <class 'numpy.object_'>, 'purchase_amount': <class 'numpy.float64'>, 'signature_provided': <class 'numpy.float64'>, 'merchant_subsector_description': 'str', 'card_fico_score': <class 'numpy.float64'>, 'card_age': <class 'numpy.float64'>, 'merchant_state': 'str'}
[2022-11-16 15:58:55,989] [2071026/MainThread] [INFO] [root] Loaded table
[2022-11-16 15:58:55,990] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0      2017-01-01 00:39:57
1      2017-01-01 00:54:29
2      2017-01-01 01:08:26
3      2017-01-01 01:33:33
4      2017-01-01 03:31:44
5      2017-01-01 05:59:29
6      2017-01-01 08:18:54
7      2017-01-01 09:26:18
8      2017-01-01 10:36:01
9      2017-01-01 10:53:40
10     2017-01-01 12:00:57
11     2017-01-01 12:12:03
12     2017-01-01 12:21:32
13     2017-01-01 12:53:15
14     2017-01-01 12:55:21
15     2017-01-01 13:23:20
16     2017-01-01 13:34:28
17     2017-01-01 13:36:22
18     2017-01-01 13:45:55
19     2017-01-01 14:42:02
20     2017-01-01 14:59:47
21     2017-01-01 15:29:45
22     2017-01-01 18:44:14
23     2017-01-01 19:04:37
24     2017-01-01 20:25:24
25     2017-01-01 21:42:03
26     2017-01-01 22:18:01
27     2017-01-01 22:58:02
28     2017-01-02 00:53:16
29     2017-01-02 00:56:18
               ...        
7883   2017-03-31 20:15:02
7884   2017-03-31 20:26:02
7885   2017-03-31 20:31:35
7886   2017-03-31 20:31:49
7887   2017-03-31 20:34:23
7888   2017-03-31 20:43:46
7889   2017-03-31 20:44:40
7890   2017-03-31 20:45:10
7891   2017-03-31 20:48:14
7892   2017-03-31 21:13:47
7893   2017-03-31 21:15:12
7894   2017-03-31 21:30:06
7895   2017-03-31 21:32:07
7896   2017-03-31 21:32:47
7897   2017-03-31 21:41:20
7898   2017-03-31 21:50:28
7899   2017-03-31 21:58:11
7900   2017-03-31 21:58:15
7901   2017-03-31 21:59:59
7902   2017-03-31 22:14:33
7903   2017-03-31 22:22:47
7904   2017-03-31 22:27:39
7905   2017-03-31 22:29:30
7906   2017-03-31 22:34:10
7907   2017-03-31 22:34:15
7908   2017-03-31 23:04:25
7909   2017-03-31 23:05:40
7910   2017-03-31 23:18:24
7911   2017-03-31 23:27:10
7912   2017-03-31 23:54:02
Name: purchase_date_parsed, Length: 7913, dtype: datetime64[ns]
[2022-11-16 15:58:55,992] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0       3.692220e+09
1       3.692221e+09
2       3.692222e+09
3       3.692223e+09
4       3.692230e+09
5       3.692239e+09
6       3.692248e+09
7       3.692252e+09
8       3.692256e+09
9       3.692257e+09
10      3.692261e+09
11      3.692262e+09
12      3.692262e+09
13      3.692264e+09
14      3.692264e+09
15      3.692266e+09
16      3.692266e+09
17      3.692267e+09
18      3.692267e+09
19      3.692271e+09
20      3.692272e+09
21      3.692273e+09
22      3.692285e+09
23      3.692286e+09
24      3.692291e+09
25      3.692296e+09
26      3.692298e+09
27      3.692300e+09
28      3.692307e+09
29      3.692307e+09
            ...     
7883    3.699980e+09
7884    3.699981e+09
7885    3.699981e+09
7886    3.699981e+09
7887    3.699981e+09
7888    3.699982e+09
7889    3.699982e+09
7890    3.699982e+09
7891    3.699982e+09
7892    3.699984e+09
7893    3.699984e+09
7894    3.699985e+09
7895    3.699985e+09
7896    3.699985e+09
7897    3.699985e+09
7898    3.699986e+09
7899    3.699986e+09
7900    3.699986e+09
7901    3.699986e+09
7902    3.699987e+09
7903    3.699988e+09
7904    3.699988e+09
7905    3.699988e+09
7906    3.699988e+09
7907    3.699988e+09
7908    3.699990e+09
7909    3.699990e+09
7910    3.699991e+09
7911    3.699992e+09
7912    3.699993e+09
Name: purchase_date_parsed, Length: 7913, dtype: float64
[2022-11-16 15:58:55,994] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0      2020-12-01
1      2020-02-01
2      2020-04-01
3      2020-11-01
4      2020-02-01
5      2019-03-01
6      2018-04-01
7      2020-12-01
8      2020-10-01
9      2019-06-01
10     2020-08-01
11     2020-11-01
12     2018-12-01
13     2020-06-01
14     2020-08-01
15     2020-08-01
16     2020-04-01
17     2020-09-01
18     2020-09-01
19     2020-12-01
20     2020-06-01
21     2020-04-01
22     2020-08-01
23     2020-07-01
24     2019-10-01
25     2020-09-01
26     2020-10-01
27     2019-10-01
28     2020-10-01
29     2020-12-01
          ...    
7883   2020-08-01
7884   2020-09-01
7885   2020-08-01
7886   2021-01-01
7887   2021-02-01
7888   2018-09-01
7889   2019-06-01
7890   2018-11-01
7891   2020-12-01
7892   2021-03-01
7893   2021-02-01
7894   2020-07-01
7895   2020-03-01
7896   2020-09-01
7897   2020-06-01
7898   2020-05-01
7899   2020-08-01
7900   2020-10-01
7901   2020-09-01
7902   2020-06-01
7903   2020-11-01
7904   2021-03-01
7905   2020-10-01
7906   2019-07-01
7907   2020-04-01
7908   2021-03-01
7909   2018-12-01
7910   2020-11-01
7911   2021-03-01
7912   2020-10-01
Name: card_first_active_month, Length: 7913, dtype: datetime64[ns]
[2022-11-16 15:58:55,995] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0       3.815770e+09
1       3.789504e+09
2       3.794688e+09
3       3.813178e+09
4       3.789504e+09
5       3.760387e+09
6       3.731530e+09
7       3.815770e+09
8       3.810499e+09
9       3.768336e+09
10      3.805229e+09
11      3.813178e+09
12      3.752611e+09
13      3.799958e+09
14      3.805229e+09
15      3.805229e+09
16      3.794688e+09
17      3.807907e+09
18      3.807907e+09
19      3.815770e+09
20      3.799958e+09
21      3.794688e+09
22      3.805229e+09
23      3.802550e+09
24      3.778877e+09
25      3.807907e+09
26      3.810499e+09
27      3.778877e+09
28      3.810499e+09
29      3.815770e+09
            ...     
7883    3.805229e+09
7884    3.807907e+09
7885    3.805229e+09
7886    3.818448e+09
7887    3.821126e+09
7888    3.744749e+09
7889    3.768336e+09
7890    3.750019e+09
7891    3.815770e+09
7892    3.823546e+09
7893    3.821126e+09
7894    3.802550e+09
7895    3.792010e+09
7896    3.807907e+09
7897    3.799958e+09
7898    3.797280e+09
7899    3.805229e+09
7900    3.810499e+09
7901    3.807907e+09
7902    3.799958e+09
7903    3.813178e+09
7904    3.823546e+09
7905    3.810499e+09
7906    3.770928e+09
7907    3.794688e+09
7908    3.823546e+09
7909    3.752611e+09
7910    3.813178e+09
7911    3.823546e+09
7912    3.810499e+09
Name: card_first_active_month, Length: 7913, dtype: float64
[2022-11-16 15:58:55,996] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Coercion done
[2022-11-16 15:58:55,997] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] Adding diagnostic: <DoctorDiagnostic ML_DIAGNOSTICS_DATASET_SANITY_CHECKS: The test dataset is imbalanced (balance=0.49), metrics can be misleading>
[2022-11-16 15:58:55,997] [2071026/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded test df: shape=(7913,35)
[2022-11-16 15:58:55,997] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading test set
[2022-11-16 15:58:55,997] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Collecting statistics
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at date... (type=CATEGORY)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_latitude... (type=NUMERIC)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at spending_type... (type=CATEGORY)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_day... (type=NUMERIC)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_month... (type=NUMERIC)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_fico_score... (type=NUMERIC)
[2022-11-16 15:58:55,998] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2022-11-16 15:58:56,000] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_id... (type=CATEGORY)
[2022-11-16 15:58:56,000] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_longitude... (type=NUMERIC)
[2022-11-16 15:58:56,000] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at item_category... (type=CATEGORY)
[2022-11-16 15:58:56,000] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at dist_cardholder_merchant... (type=NUMERIC)
[2022-11-16 15:58:56,000] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_state_enName... (type=CATEGORY)
[2022-11-16 15:58:56,001] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_date_parsed... (type=NUMERIC)
[2022-11-16 15:58:56,001] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_state... (type=CATEGORY)
[2022-11-16 15:58:56,001] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_state_enName... (type=CATEGORY)
[2022-11-16 15:58:56,001] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at signature_provided... (type=NUMERIC)
[2022-11-16 15:58:56,001] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2022-11-16 15:58:56,002] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_dow... (type=NUMERIC)
[2022-11-16 15:58:56,002] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_age... (type=NUMERIC)
[2022-11-16 15:58:56,002] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at transaction_id... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_year... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_first_active_month... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at days_active... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_latitude... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_category_id... (type=NUMERIC)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_id... (type=CATEGORY)
[2022-11-16 15:58:56,004] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_subsector_description... (type=CATEGORY)
[2022-11-16 15:58:56,010] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_longitude... (type=NUMERIC)
[2022-11-16 15:58:56,010] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_hour... (type=NUMERIC)
[2022-11-16 15:58:56,010] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_geopoint... (type=CATEGORY)
[2022-11-16 15:58:56,010] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_amount... (type=NUMERIC)
[2022-11-16 15:58:56,010] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2022-11-16 15:58:56,012] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at card_reward_program... (type=CATEGORY)
[2022-11-16 15:58:56,012] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_state... (type=CATEGORY)
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at merchant_geopoint... (type=CATEGORY)
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at risk... (type=CATEGORY)
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at purchase_date... (type=CATEGORY)
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at authorized_flag... (type=NUMERIC)
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Collecting statistics
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] generating interactions
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] {'feature_selection_params': {'method': 'NONE', 'random_forest_params': {'n_trees': 30, 'depth': 10, 'n_features': 25}, 'lasso_params': {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], 'cross_validate': True}, 'pca_params': {'n_features': 25, 'variance_proportion': 0.9}, 'correlation_params': {'min_abs_correlation': 0.0, 'max_abs_correlation': 1.0, 'n_features': 25}, 'custom_params': {'code': '# type your code here'}}, 'preprocessingFitSampleRatio': 1.0, 'preprocessingFitSampleSeed': 1337, 'target_remapping': [{'sourceValue': '0', 'mappedValue': 0, 'sampleFreq': 944}, {'sourceValue': '1', 'mappedValue': 1, 'sampleFreq': 9056}], 'skipPreprocessing': False, 'per_feature': {'date': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DateSource'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_latitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'spending_type': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_day': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_month': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_fico_score': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_id': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_longitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'item_category': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'dist_cardholder_merchant': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_state_enName': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_date_parsed': {'generate_derivative': False, 'numerical_handling': 'DATETIME_CYCLICAL', 'missing_handling': 'DROP_ROW', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': ['DAY', 'HOUR', 'MINUTE', 'WEEK'], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Date'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_state': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_state_enName': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'signature_provided': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_dow': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_age': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'transaction_id': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_IDENTIFIER', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_year': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'autoReason': 'REJECT_ZERO_VARIANCE', 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_first_active_month': {'generate_derivative': False, 'numerical_handling': 'DATETIME_CYCLICAL', 'missing_handling': 'DROP_ROW', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': ['QUARTER', 'WEEK', 'YEAR'], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Date'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'days_active': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_latitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_category_id': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_id': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_subsector_description': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_longitude': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_hour': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'REJECT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_geopoint': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'GeoPoint'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_amount': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'card_reward_program': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_state': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'USStateMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'merchant_geopoint': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'GeoPoint'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'risk': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'purchase_date': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'REJECT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DateSource'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'authorized_flag': {'generate_derivative': False, 'impute_constant_value': 0.0, 'keep_regular': False, 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'TARGET', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}}, 'reduce': {'enabled': False, 'kept_variance': 0.0}, 'feature_generation': {'pairwise_linear': {'behavior': 'DISABLED'}, 'polynomial_combinations': {'behavior': 'DISABLED'}, 'manual_interactions': {'interactions': []}, 'numericals_clustering': {'k': 0, 'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}, 'categoricals_count_transformer': {'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}}}
[2022-11-16 15:58:56,018] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] No feature selection to perform
[2022-11-16 15:58:56,019] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing train set
[2022-11-16 15:58:56,019] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 31743
[2022-11-16 15:58:56,019] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RemapValueToOutput
[2022-11-16 15:58:56,022] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,022] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {'card_fico_score': 567.6713291119302, 'signature_provided': 0.18835648804460825, 'card_age': 45.99971647292316, 'purchase_amount': 220.34435056548173}
[2022-11-16 15:58:56,024] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,024] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,024] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_fico_score)
[2022-11-16 15:58:56,024] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_fico_score (avg=567.6713291119302 std=166.93109653123216 shift=567.6713291119302 inv_scale=0.005990495604351966)
[2022-11-16 15:58:56,036] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_fico_score (avg=1.4877471190019936e-16 std=0.9999999999999861) nulls=0
[2022-11-16 15:58:56,036] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (signature_provided)
[2022-11-16 15:58:56,036] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale signature_provided (avg=0.18835648804460825 std=0.3910027336666669 shift=0.18835648804460825 inv_scale=2.557526876148412)
[2022-11-16 15:58:56,038] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled signature_provided (avg=3.961939444396827e-16 std=1.0000000000000433) nulls=0
[2022-11-16 15:58:56,038] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (card_age)
[2022-11-16 15:58:56,038] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_age (avg=45.99971647292316 std=20.905981872409836 shift=45.99971647292316 inv_scale=0.04783319942124918)
[2022-11-16 15:58:56,039] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_age (avg=4.82030486261031e-17 std=1.0000000000000113) nulls=0
[2022-11-16 15:58:56,039] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (purchase_amount)
[2022-11-16 15:58:56,040] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount (avg=220.34435056548173 std=497.31930964818537 shift=220.34435056548173 inv_scale=0.0020107805600941215)
[2022-11-16 15:58:56,041] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount (avg=-5.3224951495012104e-15 std=1.0000000000000229) nulls=0
[2022-11-16 15:58:56,041] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2022-11-16 15:58:56,041] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(datetime_cyclical)
[2022-11-16 15:58:56,041] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_subsector_description)
[2022-11-16 15:58:56,051] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(31743, 40) nnz=31743
[2022-11-16 15:58:56,051] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state)
[2022-11-16 15:58:56,060] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(31743, 13) nnz=31743
[2022-11-16 15:58:56,060] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,060] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,060] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(interaction)
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RealignTarget
[2022-11-16 15:58:56,061] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (31743,)
[2022-11-16 15:58:56,062] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (31743,)
[2022-11-16 15:58:56,062] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DropRowsWhereNoTarget
[2022-11-16 15:58:56,063] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because one of ['target'] is missing or infinity
[2022-11-16 15:58:56,063] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (31743, 57)
[2022-11-16 15:58:56,063] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] target before = (31743,)
[2022-11-16 15:58:56,066] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(31743, 35)
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (31743, 57)
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] target after = (31743,)
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (31743, 35) 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(31743, 57) 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((31743,))
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:EmitCurrentMFAsResult
[2022-11-16 15:58:56,085] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 31743
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (31743, 35) 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((31743,))
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((31743, 57))
[2022-11-16 15:58:56,085] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((31743, 35))
[2022-11-16 15:58:56,085] [2071026/MainThread] [INFO] [root] train multiframe initial size: 31743
[2022-11-16 15:58:56,086] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing train set
[2022-11-16 15:58:56,087] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing test set
[2022-11-16 15:58:56,087] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 7913
[2022-11-16 15:58:56,087] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RemapValueToOutput
[2022-11-16 15:58:56,088] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,088] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {'card_fico_score': 567.6713291119302, 'signature_provided': 0.18835648804460825, 'card_age': 45.99971647292316, 'purchase_amount': 220.34435056548173}
[2022-11-16 15:58:56,089] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,089] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,089] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_fico_score)
[2022-11-16 15:58:56,090] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_fico_score (avg=566.9442689245545 std=167.78382093957782 shift=567.6713291119302 inv_scale=0.005990495604351966)
[2022-11-16 15:58:56,090] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_fico_score (avg=-0.004355450856573021 std=1.0051082418199193) nulls=0
[2022-11-16 15:58:56,090] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (signature_provided)
[2022-11-16 15:58:56,091] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale signature_provided (avg=0.18551750284342222 std=0.38874137501866035 shift=0.18835648804460825 inv_scale=2.557526876148412)
[2022-11-16 15:58:56,091] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled signature_provided (avg=-0.007260780953020833 std=0.994216514481171) nulls=0
[2022-11-16 15:58:56,091] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (card_age)
[2022-11-16 15:58:56,091] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale card_age (avg=46.03298369771262 std=20.861750595863633 shift=45.99971647292316 inv_scale=0.04783319942124918)
[2022-11-16 15:58:56,092] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled card_age (avg=0.0015912777975458684 std=0.997884276528308) nulls=0
[2022-11-16 15:58:56,092] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (purchase_amount)
[2022-11-16 15:58:56,092] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale purchase_amount (avg=221.107731580942 std=559.0600701562392 shift=220.34435056548173 inv_scale=0.0020107805600941215)
[2022-11-16 15:58:56,093] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled purchase_amount (avg=0.0015349917058339014 std=1.1241471209950162) nulls=0
[2022-11-16 15:58:56,093] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2022-11-16 15:58:56,093] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(datetime_cyclical)
[2022-11-16 15:58:56,093] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_subsector_description)
[2022-11-16 15:58:56,096] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(7913, 40) nnz=7913
[2022-11-16 15:58:56,096] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (merchant_state)
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(7913, 13) nnz=7913
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(interaction)
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RealignTarget
[2022-11-16 15:58:56,099] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (7913,)
[2022-11-16 15:58:56,100] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (7913,)
[2022-11-16 15:58:56,100] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DropRowsWhereNoTarget
[2022-11-16 15:58:56,100] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because one of ['target'] is missing or infinity
[2022-11-16 15:58:56,100] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (7913, 57)
[2022-11-16 15:58:56,100] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] target before = (7913,)
[2022-11-16 15:58:56,101] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2022-11-16 15:58:56,107] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(7913, 35)
[2022-11-16 15:58:56,107] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (7913, 57)
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] target after = (7913,)
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (7913, 35) 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(7913, 57) 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((7913,))
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:EmitCurrentMFAsResult
[2022-11-16 15:58:56,108] [2071026/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 7913
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (7913, 35) 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((7913,))
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((7913, 57))
[2022-11-16 15:58:56,108] [2071026/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((7913, 35))
[2022-11-16 15:58:56,108] [2071026/MainThread] [INFO] [root] test multiframe initial size: 7913
[2022-11-16 15:58:56,108] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing test set
[2022-11-16 15:58:56,109] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Hyperparameter searching
[2022-11-16 15:58:56,109] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(31743,57) tn=1809351 nnz=190458 fill_ratio=0.11
[2022-11-16 15:58:56,109] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] too small, using array
[2022-11-16 15:58:56,122] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Create CLF from params: {'algorithm': 'RANDOM_FOREST_CLASSIFICATION', 'rf_classifier_grid': {'n_estimators': {'gridMode': 'EXPLICIT', 'randomMode': 'RANGE', 'limit': {'min': 1}, 'values': [100], 'range': {'min': 80, 'max': 200, 'nbValues': 3, 'scaling': 'LINEAR'}}, 'n_jobs': 4, 'max_tree_depth': {'gridMode': 'EXPLICIT', 'randomMode': 'RANGE', 'limit': {'min': 1}, 'values': [6, 15], 'range': {'min': 6, 'max': 20, 'nbValues': 3, 'scaling': 'LINEAR'}}, 'min_samples_leaf': {'gridMode': 'EXPLICIT', 'randomMode': 'RANGE', 'limit': {'min': 1}, 'values': [1], 'range': {'min': 1, 'max': 20, 'nbValues': 3, 'scaling': 'LINEAR'}}, 'selection_mode': 'auto', 'max_features': {'gridMode': 'EXPLICIT', 'randomMode': 'RANGE', 'limit': {'min': 1}, 'values': [5], 'range': {'min': 1, 'max': 20, 'nbValues': 3, 'scaling': 'LINEAR'}}, 'max_feature_prop': {'gridMode': 'EXPLICIT', 'randomMode': 'RANGE', 'limit': {'min': 1e-23, 'max': 1.0}, 'values': [0.3], 'range': {'min': 0.1, 'max': 0.7, 'nbValues': 3, 'scaling': 'LINEAR'}}, 'enabled': True}, 'max_ensemble_nodes_serialized': 6000, 'metrics': {'customMetrics': [], 'evaluationMetric': 'ROC_AUC', 'customEvaluationMetricGIB': True, 'customEvaluationMetricNeedsProba': False, 'thresholdOptimizationMetric': 'F1', 'costMatrixWeights': {'tpGain': 1.0, 'tnGain': 0.0, 'fpGain': -0.3, 'fnGain': 0.0}, 'liftPoint': 0.4, 'unitaryTreatmentCost': 0.0, 'unitaryConversionBenefit': 0.0}, 'autoOptimizeThreshold': True, 'forcedClassifierThreshold': 0.0, 'gridLength': 2, 'grid_search_params': {'mode': 'KFOLD', 'splitRatio': 0.8, 'shuffleIterations': 1, 'nFolds': 5, 'cvSeed': 1337, 'stratified': True, 'strategy': 'GRID', 'bayesianOptimizer': 'SCIKIT_OPTIMIZE', 'randomized': True, 'seed': 1337, 'nIter': 0, 'nIterRandom': 5, 'timeout': 0, 'nJobs': 4, 'distributed': False, 'nContainers': 4, 'foldOffset': True}, 'pluginAlgoCustomGridSearch': False, 'computeLearningCurves': False, 'skipExpensiveReports': False} for algorithm RANDOM_FOREST_CLASSIFICATION
[2022-11-16 15:58:56,122] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Using stratified K-Fold CV with k=5
[2022-11-16 15:58:56,128] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluation_monitor] No distributed container configuration is available to run this search
[2022-11-16 15:58:56,136] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Fitting 5 folds for each of 2 candidates, totalling 10 fits
[2022-11-16 15:58:56,136] [2071026/MainThread] [INFO] [dataiku.base.utils] Detected ?/8 usable CPUs
[2022-11-16 15:58:56,136] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Execute hyperparameter search locally on 4 threads
[2022-11-16 15:58:56,136] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.strategies.grid_search_strategy] Running GridSearchStrategy for hyperparameters space: <dataiku.doctor.prediction.common.TreesHyperparametersSpace object at 0x7facd03f60b8>
[2022-11-16 15:58:56,137] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:56,137] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-0
[2022-11-16 15:58:56,137] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-0
[2022-11-16 15:58:56,137] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T1
[2022-11-16 15:58:56,137] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:56,137] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T1 on local-0...
[2022-11-16 15:58:56,137] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-1
[2022-11-16 15:58:56,137] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T2
[2022-11-16 15:58:56,137] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=0: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:56,137] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-1
[2022-11-16 15:58:56,137] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:56,138] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T2 on local-1...
[2022-11-16 15:58:56,138] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-2
[2022-11-16 15:58:56,138] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Fit s=1: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:56,138] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T3
[2022-11-16 15:58:56,138] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-2
[2022-11-16 15:58:56,139] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 3: {'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:56,139] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T3 on local-2...
[2022-11-16 15:58:56,139] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=2: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:56,139] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-3
[2022-11-16 15:58:56,139] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T4
[2022-11-16 15:58:56,139] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-3
[2022-11-16 15:58:56,139] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 4: {'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:56,139] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T4 on local-3...
[2022-11-16 15:58:56,140] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=3: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:58:59,720] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=2: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=3.2s st=0.4s sc=0.7008682877241048, sg=1)
[2022-11-16 15:58:59,721] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T3 done
[2022-11-16 15:58:59,722] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T5
[2022-11-16 15:58:59,722] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T5 on local-2...
[2022-11-16 15:58:59,722] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=4: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:59,722] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:58:59,901] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=0: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=3.3s st=0.5s sc=0.7109440403579155, sg=1)
[2022-11-16 15:58:59,902] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T1 done
[2022-11-16 15:58:59,902] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T6
[2022-11-16 15:58:59,902] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:59,902] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T6 on local-0...
[2022-11-16 15:58:59,903] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=0: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:59,906] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Done s=1: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=3.3s st=0.5s sc=0.7054323363449558, sg=1)
[2022-11-16 15:58:59,907] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T2 done
[2022-11-16 15:58:59,908] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T7
[2022-11-16 15:58:59,908] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:59,913] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T7 on local-1...
[2022-11-16 15:58:59,913] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Fit s=1: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:58:59,921] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=3: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=3.3s st=0.5s sc=0.7070731392593624, sg=1)
[2022-11-16 15:58:59,922] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T4 done
[2022-11-16 15:58:59,923] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T8
[2022-11-16 15:58:59,923] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 3: {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:58:59,923] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T8 on local-3...
[2022-11-16 15:58:59,924] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=2: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:59:02,803] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=4: max_depth=6, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=1.9s st=1.2s sc=0.7016673490543849, sg=1)
[2022-11-16 15:59:02,803] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T5 done
[2022-11-16 15:59:02,803] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T9
[2022-11-16 15:59:02,804] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T9 on local-2...
[2022-11-16 15:59:02,804] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=3: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:59:02,805] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 4: {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:59:03,637] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=2: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=2.8s st=0.9s sc=0.6985825260662991, sg=1)
[2022-11-16 15:59:03,639] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T8 done
[2022-11-16 15:59:03,639] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Enqueued task T10
[2022-11-16 15:59:03,640] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task T10 on local-3...
[2022-11-16 15:59:03,640] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=4: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 
[2022-11-16 15:59:03,766] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Done s=1: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=2.8s st=1.0s sc=0.7042825923872272, sg=1)
[2022-11-16 15:59:03,766] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T7 done
[2022-11-16 15:59:03,993] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=0: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=2.9s st=1.2s sc=0.7157601900475119, sg=1)
[2022-11-16 15:59:03,993] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T6 done
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:59:04,819] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=3: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=1.4s st=0.6s sc=0.7097113118831287, sg=1)
[2022-11-16 15:59:04,819] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T9 done
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/crossval/search_context.py:346: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(scorer)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python36.packages/sklearn/ensemble/base.py:154: UserWarning: Loky-backed parallel loops cannot be nested below threads, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[2022-11-16 15:59:05,191] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=4: max_depth=15, max_features=auto, min_samples_leaf=1, n_estimators=100, min_samples_split=3 (ft=1.2s st=0.3s sc=0.699480668464105, sg=1)
[2022-11-16 15:59:05,192] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task T10 done
[2022-11-16 15:59:05,199] [2071026/Thread-1] [INFO] [dataiku.doctor.crossval.search_runner] Completed search for hyperparameters
[2022-11-16 15:59:05,199] [2071026/Thread-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Scheduler has been soft interrupted
[2022-11-16 15:59:05,200] [2071026/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Scheduler has been hard interrupted (shutdown)
[2022-11-16 15:59:05,200] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-1
[2022-11-16 15:59:05,201] [2071026/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-1
[2022-11-16 15:59:05,201] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-2
[2022-11-16 15:59:05,201] [2071026/Thread-5:local-3] [ERROR] [dataiku.doctor.distributed.work_scheduler] All workers are dead, interrupt the scheduler
[2022-11-16 15:59:05,200] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-0
[2022-11-16 15:59:05,201] [2071026/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-2
[2022-11-16 15:59:05,201] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-3
[2022-11-16 15:59:05,201] [2071026/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-3
[2022-11-16 15:59:05,201] [2071026/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-0
[2022-11-16 15:59:05,201] [2071026/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Hyperparameter search done, best_parameters being : {'max_depth': 15, 'max_features': 'auto', 'min_samples_leaf': 1, 'n_estimators': 100, 'min_samples_split': 3}
[2022-11-16 15:59:05,202] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Hyperparameter searching
[2022-11-16 15:59:05,202] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Fitting model
[2022-11-16 15:59:05,205] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Fitting model:
[2022-11-16 15:59:05,206] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common]   Model is: RandomForestClassifier(bootstrap=True,
            class_weight={0: 4.9244492708656535, 1: 0.5565042075736325},
            criterion='gini', max_depth=15, max_features='auto',
            max_leaf_nodes=None, min_impurity_decrease=0.0,
            min_impurity_split=None, min_samples_leaf=1,
            min_samples_split=3, min_weight_fraction_leaf=0.0,
            n_estimators=100, n_jobs=4, oob_score=False, random_state=1337,
            verbose=0, warm_start=False)
[2022-11-16 15:59:05,206] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X class: <class 'numpy.ndarray'>
[2022-11-16 15:59:05,206] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X shape: (31743, 57)
[2022-11-16 15:59:05,206] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_y shape: (31743,)
[2022-11-16 15:59:06,269] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_fit] Obtained RF CLF params: {'bootstrap': True, 'class_weight': {0: 4.9244492708656535, 1: 0.5565042075736325}, 'criterion': 'gini', 'max_depth': 15, 'max_features': 'auto', 'max_leaf_nodes': None, 'min_impurity_decrease': 0.0, 'min_impurity_split': None, 'min_samples_leaf': 1, 'min_samples_split': 3, 'min_weight_fraction_leaf': 0.0, 'n_estimators': 100, 'n_jobs': 4, 'oob_score': False, 'random_state': 1337, 'verbose': 0, 'warm_start': False} 
[2022-11-16 15:59:06,269] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_fit] Output params are {'resolved': {'algorithm': 'RANDOM_FOREST_CLASSIFICATION', 'skipExpensiveReports': False, 'rf': {'estimators': 100, 'max_tree_depth': 15, 'min_samples_leaf': 1, 'selection_mode': 'auto'}}, 'other': {'rf_min_samples_split': 3}}
[2022-11-16 15:59:06,482] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Fitting model
[2022-11-16 15:59:06,483] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Saving model
[2022-11-16 15:59:06,483] [2071026/MainThread] [INFO] [dataiku.doctor.prediction_entrypoints] PICKLING RandomForestClassifier(bootstrap=True,
            class_weight={0: 4.9244492708656535, 1: 0.5565042075736325},
            criterion='gini', max_depth=15, max_features='auto',
            max_leaf_nodes=None, min_impurity_decrease=0.0,
            min_impurity_split=None, min_samples_leaf=1,
            min_samples_split=3, min_weight_fraction_leaf=0.0,
            n_estimators=100, n_jobs=4, oob_score=False, random_state=1337,
            verbose=0, warm_start=False)
[2022-11-16 15:59:06,533] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Saving model
[2022-11-16 15:59:06,533] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Scoring model
[2022-11-16 15:59:06,533] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Intrinsic scoring
[2022-11-16 15:59:06,533] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Computing feature importance
[2022-11-16 15:59:06,736] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.linear_coefficients_computation] Not computing linear coefficients because attribute `coef_` is not present
[2022-11-16 15:59:06,736] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Extracting rescalers
[2022-11-16 15:59:06,736] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Creating random forest trees summary
[2022-11-16 15:59:06,830] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(7913,57) tn=451041 nnz=47478 fill_ratio=0.11
[2022-11-16 15:59:06,830] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] too small, using array
[2022-11-16 15:59:07,044] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Probas raw shape 7913/2 target_map=2
[2022-11-16 15:59:07,121] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Starting threshold optim
[2022-11-16 15:59:07,123] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.000000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,125] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.025000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,127] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.050000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,129] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.075000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,131] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.100000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,133] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.125000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,134] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.150000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,136] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.175000 score 0.944089 (pred_true=7913)
[2022-11-16 15:59:07,138] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.200000 score 0.944215 (pred_true=7911)
[2022-11-16 15:59:07,140] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.225000 score 0.944341 (pred_true=7909)
[2022-11-16 15:59:07,142] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.250000 score 0.944467 (pred_true=7907)
[2022-11-16 15:59:07,144] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.275000 score 0.944719 (pred_true=7903)
[2022-11-16 15:59:07,145] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.300000 score 0.944548 (pred_true=7893)
[2022-11-16 15:59:07,147] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.325000 score 0.943894 (pred_true=7861)
[2022-11-16 15:59:07,149] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.350000 score 0.942291 (pred_true=7810)
[2022-11-16 15:59:07,151] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.375000 score 0.939316 (pred_true=7723)
[2022-11-16 15:59:07,152] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.400000 score 0.931642 (pred_true=7510)
[2022-11-16 15:59:07,153] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.425000 score 0.907852 (pred_true=7011)
[2022-11-16 15:59:07,155] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.450000 score 0.849529 (pred_true=5984)
[2022-11-16 15:59:07,156] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.475000 score 0.765241 (pred_true=4801)
[2022-11-16 15:59:07,158] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.500000 score 0.702353 (pred_true=4059)
[2022-11-16 15:59:07,159] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.525000 score 0.665059 (pred_true=3706)
[2022-11-16 15:59:07,161] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.550000 score 0.642270 (pred_true=3500)
[2022-11-16 15:59:07,162] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.575000 score 0.628336 (pred_true=3378)
[2022-11-16 15:59:07,164] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.600000 score 0.617536 (pred_true=3292)
[2022-11-16 15:59:07,165] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.625000 score 0.602967 (pred_true=3171)
[2022-11-16 15:59:07,167] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.650000 score 0.585699 (pred_true=3036)
[2022-11-16 15:59:07,168] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.675000 score 0.558862 (pred_true=2838)
[2022-11-16 15:59:07,170] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.700000 score 0.526142 (pred_true=2603)
[2022-11-16 15:59:07,171] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.725000 score 0.488478 (pred_true=2342)
[2022-11-16 15:59:07,173] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.750000 score 0.458871 (pred_true=2152)
[2022-11-16 15:59:07,174] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.775000 score 0.424928 (pred_true=1943)
[2022-11-16 15:59:07,176] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.800000 score 0.387551 (pred_true=1729)
[2022-11-16 15:59:07,177] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.825000 score 0.347543 (pred_true=1511)
[2022-11-16 15:59:07,179] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.850000 score 0.302406 (pred_true=1278)
[2022-11-16 15:59:07,180] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.875000 score 0.239344 (pred_true=972)
[2022-11-16 15:59:07,182] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.900000 score 0.166516 (pred_true=648)
[2022-11-16 15:59:07,183] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.925000 score 0.088319 (pred_true=330)
[2022-11-16 15:59:07,184] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.950000 score 0.042335 (pred_true=153)
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] AT CUT 0.975000 score 0.004794 (pred_true=17)
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Selected threshold 0.275 
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] preds [1 1 1 ... 1 1 1]
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MAPPED SHAPE 7913
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] k=0 v=0
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Mask data [False False False ... False False False]
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] mapped pred <class 'numpy.ndarray'>
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] k=1 v=1
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Mask data [ True  True  True ...  True  True  True]
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] mapped pred <class 'numpy.ndarray'>
[2022-11-16 15:59:07,186] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MAPPED PREDS ['1' '1' '1' ... '1' '1' '1']
[2022-11-16 15:59:07,193] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Total true rate = 7075 / 7913
[2022-11-16 15:59:07,193] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Computing lift on first 3165 rows
[2022-11-16 15:59:07,194] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] True rate on considered : 3165 / 3083
[2022-11-16 15:59:07,194] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.common] Lift = 1.089468
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Density for 0 (id 0)
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] valid_y shape = (7913,)
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] CP: <class 'numpy.ndarray'> ((7913,))
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Actual shape = (7913,)
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MASK is <class 'numpy.ndarray'> 
[2022-11-16 15:59:07,216] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba shape (838,)
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Density for 1 (id 1)
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] valid_y shape = (7913,)
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] CP: <class 'numpy.ndarray'> ((7913,))
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Actual shape = (7913,)
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] MASK is <class 'numpy.ndarray'> 
[2022-11-16 15:59:07,255] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Class proba shape (7075,)
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/utils/calibration.py:34: RuntimeWarning: invalid value encountered in true_divide
  freq_pos = y_true[i_start:i_end].sum() / weight
/data/home/dkuacademy/Applications/dataiku-dss-11.1.0/python/dataiku/doctor/utils/calibration.py:35: RuntimeWarning: invalid value encountered in double_scalars
  avg_pred = y_prob[i_start:i_end].sum() / weight
[2022-11-16 15:59:07,905] [2071026/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] Adding diagnostic: <DoctorDiagnostic ML_DIAGNOSTICS_ABNORMAL_PREDICTIONS_DETECTION: The model almost always predicts the class '1'>
[2022-11-16 15:59:07,905] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Scoring model
[2022-11-16 15:59:07,905] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preparing explainability
[2022-11-16 15:59:07,905] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Start actual predict (threshold=0.5)
[2022-11-16 15:59:08,008] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Done actual predict
[2022-11-16 15:59:08,008] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.classification_scoring] Probas raw shape 31743/2 target_map=2
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [root] Reading with dtypes: None
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for transaction_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for authorized_flag: <class 'numpy.object_'> (schema_type=bigint feature_type=NUMERIC feature_role=TARGET)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_date_parsed: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_month: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_day: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_dow: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_hour: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,017] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_id: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_category_id: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for item_category: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for purchase_amount: <class 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for signature_provided: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_subsector_description: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for spending_type: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_first_active_month: None (schema_type=date feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for days_active: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_reward_program: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_latitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_longitude: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_fico_score: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_age: <class 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for card_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_geopoint: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for merchant_state_enName: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dist_cardholder_merchant: None (schema_type=double feature_type=NUMERIC feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for risk: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2022-11-16 15:59:08,018] [2071026/MainThread] [INFO] [root] Reading with FIXED dtypes: {'authorized_flag': <class 'numpy.object_'>, 'purchase_amount': <class 'numpy.float64'>, 'signature_provided': <class 'numpy.float64'>, 'merchant_subsector_description': 'str', 'card_fico_score': <class 'numpy.float64'>, 'card_age': <class 'numpy.float64'>, 'merchant_state': 'str'}
[2022-11-16 15:59:08,287] [2071026/MainThread] [INFO] [root] Loaded table
[2022-11-16 15:59:08,293] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Building background rows
[2022-11-16 15:59:08,295] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Background estimated bias <= 0.021802901616053805
[2022-11-16 15:59:08,306] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Built background rows with shape=(1000, 35)
[2022-11-16 15:59:08,307] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Saving background rows with shape=(1000, 35)
[2022-11-16 15:59:08,333] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2017-01-01 00:00:59
1       2017-01-01 00:21:40
2       2017-01-01 00:58:34
3       2017-01-01 01:09:52
4       2017-01-01 01:26:59
5       2017-01-01 01:34:19
6       2017-01-01 01:35:42
7       2017-01-01 01:57:45
8       2017-01-01 02:31:18
9       2017-01-01 03:26:33
10      2017-01-01 03:32:02
11      2017-01-01 03:50:52
12      2017-01-01 05:10:03
13      2017-01-01 05:35:05
14      2017-01-01 07:22:04
15      2017-01-01 08:00:13
16      2017-01-01 08:48:45
17      2017-01-01 09:01:54
18      2017-01-01 09:12:58
19      2017-01-01 09:28:44
20      2017-01-01 09:33:51
21      2017-01-01 09:38:16
22      2017-01-01 09:47:15
23      2017-01-01 10:01:45
24      2017-01-01 10:21:58
25      2017-01-01 10:35:16
26      2017-01-01 10:35:46
27      2017-01-01 10:36:27
28      2017-01-01 10:49:58
29      2017-01-01 10:50:30
                ...        
31713   2017-03-31 23:17:09
31714   2017-03-31 23:18:22
31715   2017-03-31 23:18:26
31716   2017-03-31 23:20:19
31717   2017-03-31 23:25:24
31718   2017-03-31 23:26:43
31719   2017-03-31 23:30:41
31720   2017-03-31 23:32:25
31721   2017-03-31 23:34:33
31722   2017-03-31 23:34:45
31723   2017-03-31 23:39:16
31724   2017-03-31 23:40:24
31725   2017-03-31 23:41:09
31726   2017-03-31 23:41:30
31727   2017-03-31 23:42:24
31728   2017-03-31 23:43:18
31729   2017-03-31 23:44:03
31730   2017-03-31 23:44:45
31731   2017-03-31 23:45:23
31732   2017-03-31 23:47:45
31733   2017-03-31 23:47:53
31734   2017-03-31 23:48:58
31735   2017-03-31 23:49:00
31736   2017-03-31 23:49:35
31737   2017-03-31 23:52:13
31738   2017-03-31 23:55:05
31739   2017-03-31 23:55:35
31740   2017-03-31 23:55:37
31741   2017-03-31 23:56:53
31742   2017-03-31 23:58:07
Name: purchase_date_parsed, Length: 31743, dtype: datetime64[ns]
[2022-11-16 15:59:08,336] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.692218e+09
1        3.692219e+09
2        3.692221e+09
3        3.692222e+09
4        3.692223e+09
5        3.692223e+09
6        3.692223e+09
7        3.692225e+09
8        3.692227e+09
9        3.692230e+09
10       3.692230e+09
11       3.692231e+09
12       3.692236e+09
13       3.692238e+09
14       3.692244e+09
15       3.692246e+09
16       3.692249e+09
17       3.692250e+09
18       3.692251e+09
19       3.692252e+09
20       3.692252e+09
21       3.692252e+09
22       3.692253e+09
23       3.692254e+09
24       3.692255e+09
25       3.692256e+09
26       3.692256e+09
27       3.692256e+09
28       3.692257e+09
29       3.692257e+09
             ...     
31713    3.699991e+09
31714    3.699991e+09
31715    3.699991e+09
31716    3.699991e+09
31717    3.699992e+09
31718    3.699992e+09
31719    3.699992e+09
31720    3.699992e+09
31721    3.699992e+09
31722    3.699992e+09
31723    3.699992e+09
31724    3.699992e+09
31725    3.699992e+09
31726    3.699992e+09
31727    3.699993e+09
31728    3.699993e+09
31729    3.699993e+09
31730    3.699993e+09
31731    3.699993e+09
31732    3.699993e+09
31733    3.699993e+09
31734    3.699993e+09
31735    3.699993e+09
31736    3.699993e+09
31737    3.699993e+09
31738    3.699993e+09
31739    3.699993e+09
31740    3.699993e+09
31741    3.699993e+09
31742    3.699993e+09
Name: purchase_date_parsed, Length: 31743, dtype: float64
[2022-11-16 15:59:08,337] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalizing date to numeric : 0       2020-12-01
1       2019-12-01
2       2019-05-01
3       2020-09-01
4       2019-10-01
5       2020-05-01
6       2020-07-01
7       2020-11-01
8       2020-07-01
9       2020-06-01
10      2020-05-01
11      2019-11-01
12      2019-02-01
13      2019-08-01
14      2020-03-01
15      2018-10-01
16      2020-10-01
17      2020-10-01
18      2020-11-01
19      2020-09-01
20      2020-08-01
21      2020-08-01
22      2020-12-01
23      2020-02-01
24      2019-07-01
25      2019-11-01
26      2020-10-01
27      2020-12-01
28      2020-11-01
29      2020-10-01
           ...    
31713   2020-12-01
31714   2018-05-01
31715   2020-12-01
31716   2020-06-01
31717   2021-03-01
31718   2021-03-01
31719   2021-03-01
31720   2021-07-01
31721   2021-02-01
31722   2019-11-01
31723   2020-10-01
31724   2020-02-01
31725   2021-01-01
31726   2020-06-01
31727   2020-05-01
31728   2019-05-01
31729   2021-03-01
31730   2021-01-01
31731   2020-02-01
31732   2020-11-01
31733   2021-03-01
31734   2020-12-01
31735   2020-03-01
31736   2020-09-01
31737   2021-03-01
31738   2020-11-01
31739   2021-01-01
31740   2021-02-01
31741   2020-07-01
31742   2021-02-01
Name: card_first_active_month, Length: 31743, dtype: datetime64[ns]
[2022-11-16 15:59:08,340] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Normalized date : 0        3.815770e+09
1        3.784147e+09
2        3.765658e+09
3        3.807907e+09
4        3.778877e+09
5        3.797280e+09
6        3.802550e+09
7        3.813178e+09
8        3.802550e+09
9        3.799958e+09
10       3.797280e+09
11       3.781555e+09
12       3.757968e+09
13       3.773606e+09
14       3.792010e+09
15       3.747341e+09
16       3.810499e+09
17       3.810499e+09
18       3.813178e+09
19       3.807907e+09
20       3.805229e+09
21       3.805229e+09
22       3.815770e+09
23       3.789504e+09
24       3.770928e+09
25       3.781555e+09
26       3.810499e+09
27       3.815770e+09
28       3.813178e+09
29       3.810499e+09
             ...     
31713    3.815770e+09
31714    3.734122e+09
31715    3.815770e+09
31716    3.799958e+09
31717    3.823546e+09
31718    3.823546e+09
31719    3.823546e+09
31720    3.834086e+09
31721    3.821126e+09
31722    3.781555e+09
31723    3.810499e+09
31724    3.789504e+09
31725    3.818448e+09
31726    3.799958e+09
31727    3.797280e+09
31728    3.765658e+09
31729    3.823546e+09
31730    3.818448e+09
31731    3.789504e+09
31732    3.813178e+09
31733    3.823546e+09
31734    3.815770e+09
31735    3.792010e+09
31736    3.807907e+09
31737    3.823546e+09
31738    3.813178e+09
31739    3.818448e+09
31740    3.821126e+09
31741    3.802550e+09
31742    3.821126e+09
Name: card_first_active_month, Length: 31743, dtype: float64
[2022-11-16 15:59:08,341] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Coercion done
[2022-11-16 15:59:08,345] [2071026/MainThread] [INFO] [dataiku.doctor.posttraining.features_distribution] Computing features distribution
[2022-11-16 15:59:08,387] [2071026/MainThread] [INFO] [dataiku.doctor.posttraining.features_distribution] Saving features distribution
[2022-11-16 15:59:08,390] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Computing column importance
[2022-11-16 15:59:08,390] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Reusing the model feature importance
[2022-11-16 15:59:08,390] [2071026/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Saving column importance
[2022-11-16 15:59:08,393] [2071026/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preparing explainability
[2022-11-16 15:59:08,394] [2071026/MainThread] [INFO] [dataiku.doctor.utils] Training with python version: 3.6.8 (sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0))
[2022-11-16 15:59:08,397] [2071026/MainThread] [INFO] [dataiku.base.socket_block_link] Client closed
[2022/11/16-15:59:08.397] [MRT-246983] [INFO] [dku.block.link.interaction]  - Check result for nullity exceptionIfNull=true result=not null
[2022/11/16-15:59:08.397] [MRT-246983] [INFO] [dku.analysis.prediction]  - Training returned ok
[2022/11/16-15:59:08.397] [MRT-246983] [INFO] [dku.ml.versioning]  - Dumping version info {"trainedWithDSSVersion":"11.1.0","trainedWithDSSConfVersion":"11100","trainedWithDSSBackend":"PY_MEMORY","backendCompatibilityVersion":18} in /data/home/dkuacademy/Library/dss-11.0/design/analysis-data/MLOPS_V11_STARTER_TEST/GvRJLWvO/f8fZYP6m/sessions/s1/pp1/m1
[2022/11/16-15:59:08.398] [MRT-246983] [INFO] [dku.analysis.ml]  - preTrain={
  "algorithm": "RANDOM_FOREST_CLASSIFICATION",
  "rf_classifier_grid": {
    "n_estimators": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1
      },
      "values": [
        100
      ],
      "range": {
        "min": 80,
        "max": 200,
        "nbValues": 3,
        "scaling": "LINEAR"
      }
    },
    "n_jobs": 4,
    "max_tree_depth": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1
      },
      "values": [
        6,
        15
      ],
      "range": {
        "min": 6,
        "max": 20,
        "nbValues": 3,
        "scaling": "LINEAR"
      }
    },
    "min_samples_leaf": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1
      },
      "values": [
        1
      ],
      "range": {
        "min": 1,
        "max": 20,
        "nbValues": 3,
        "scaling": "LINEAR"
      }
    },
    "selection_mode": "auto",
    "max_features": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1
      },
      "values": [
        5
      ],
      "range": {
        "min": 1,
        "max": 20,
        "nbValues": 3,
        "scaling": "LINEAR"
      }
    },
    "max_feature_prop": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1.0E-23,
        "max": 1.0
      },
      "values": [
        0.3
      ],
      "range": {
        "min": 0.1,
        "max": 0.7,
        "nbValues": 3,
        "scaling": "LINEAR"
      }
    },
    "enabled": true
  },
  "max_ensemble_nodes_serialized": 6000,
  "metrics": {
    "customMetrics": [],
    "evaluationMetric": "ROC_AUC",
    "customEvaluationMetricGIB": true,
    "customEvaluationMetricNeedsProba": false,
    "thresholdOptimizationMetric": "F1",
    "costMatrixWeights": {
      "tpGain": 1.0,
      "tnGain": 0.0,
      "fpGain": -0.3,
      "fnGain": 0.0
    },
    "liftPoint": 0.4,
    "unitaryTreatmentCost": 0.0,
    "unitaryConversionBenefit": 0.0
  },
  "autoOptimizeThreshold": true,
  "forcedClassifierThreshold": 0.0,
  "gridLength": 2,
  "grid_search_params": {
    "mode": "KFOLD",
    "splitRatio": 0.8,
    "shuffleIterations": 1,
    "nFolds": 5,
    "cvSeed": 1337,
    "stratified": true,
    "strategy": "GRID",
    "bayesianOptimizer": "SCIKIT_OPTIMIZE",
    "randomized": true,
    "seed": 1337,
    "nIter": 0,
    "nIterRandom": 5,
    "timeout": 0,
    "nJobs": 4,
    "distributed": false,
    "nContainers": 4,
    "foldOffset": true
  },
  "pluginAlgoCustomGridSearch": false,
  "computeLearningCurves": false,
  "skipExpensiveReports": false
}
[2022/11/16-15:59:08.398] [MRT-246983] [INFO] [dku.analysis.ml]  - amp={
  "resolved": {
    "algorithm": "RANDOM_FOREST_CLASSIFICATION",
    "rf": {
      "estimators": 100,
      "max_tree_depth": 15,
      "min_samples_leaf": 1,
      "selection_mode": "auto",
      "max_features": 0,
      "max_feature_prop": 0.0
    },
    "skipExpensiveReports": false
  },
  "other": {
    "rf_min_samples_split": 3
  }
}
[2022/11/16-15:59:08.399] [MRT-246983] [WARN] [dku.kernels]  - Killing kernel python-single-command-kernel
[2022/11/16-15:59:08.400] [MRT-246983] [INFO] [dku.utils]  - Killing process PID: 2071026
[2022/11/16-15:59:08.403] [MRT-246983] [INFO] [dip.tickets]  - Destroying API ticket for analysis-ml-MLOPS_V11_STARTER_TEST-ee92CyB on behalf of sean
[2022/11/16-15:59:08.419] [KNL-python-single-command-kernel-monitor-246992] [INFO] [dku.kernels]  - Process done with code 143
[2022/11/16-15:59:08.424] [KNL-python-single-command-kernel-out-246997] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2022/11/16-15:59:08.424] [KNL-python-single-command-kernel-err-246998] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2022/11/16-15:59:08.424] [KNL-python-single-command-kernel-monitor-246992] [WARN] [dku.resource]  - stat file for pid 2071026 does not exist. Process died?
[2022/11/16-15:59:08.424] [KNL-python-single-command-kernel-monitor-246992] [DEBUG] [dku.resourceusage]  - Reporting completion of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"sean","projectKey":"MLOPS_V11_STARTER_TEST","analysisId":"GvRJLWvO","mlTaskId":"f8fZYP6m","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"7hOjVN0SpCuZLuT9","startTime":1668614334778,"localProcess":{"pid":2071026,"commandName":"/data/home/dkuacademy/Library/dss-11.0/design/bin/python","cpuUserTimeMS":25370,"cpuSystemTimeMS":880,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":26250,"cpuCurrent":3.333999600239856,"vmSizeMB":4285,"vmRSSMB":335,"vmHWMMB":335,"vmRSSAnonMB":295,"vmDataMB":957,"vmSizePeakMB":4485,"vmRSSPeakMB":335,"vmRSSTotalMBS":3155,"majorFaults":0,"childrenMajorFaults":0}}
[2022/11/16-15:59:08.425] [MRT-246983] [INFO] [dku.block.link]  - Closed socket
[2022/11/16-15:59:08.425] [MRT-246983] [INFO] [dku.block.link]  - Closed socket
[2022/11/16-15:59:08.425] [MRT-246983] [INFO] [dku.block.link]  - Closed serverSocket
[2022/11/16-15:59:08.425] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.ml.python] T-f8fZYP6m - [ct: 15234] Processing thread joined ...
[2022/11/16-15:59:08.425] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.ml.python] T-f8fZYP6m - [ct: 15234] Joining processing thread ...
[2022/11/16-15:59:08.426] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.ml.python] T-f8fZYP6m - [ct: 15235] Processing thread joined ...
[2022/11/16-15:59:08.426] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis] T-f8fZYP6m - [ct: 15235] Train done
[2022/11/16-15:59:08.426] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.prediction] T-f8fZYP6m - Train done
[2022/11/16-15:59:08.432] [FT-TrainWorkThread-zIAftYjN-246982] [INFO] [dku.analysis.trainingdetails] T-f8fZYP6m - Publishing mltask-train-done reflected event
