Does the model actually work?

This checks whether the model behind your figures has ever been right. Give it a period that has already finished: the chance of default each borrower was given at the start, and whether it defaulted by the end.

Failed1,200 borrowers · worked example

Failed on calibration. The allowance this model produces is understated.

This example is meant to fail. The book is 1,200 synthetic borrowers with one grade deliberately wrong: BB was priced at 2.00% and defaulted at 6.00%.

Notice what still passes. The ranking is sound, and the portfolio total — 3.75% observed against 2.75% predicted — reads as acceptable. That is the whole point: a portfolio-level test nets an understated grade against a conservative one and reports nothing wrong, so calibration is tested grade by grade. Load your own file above to replace this.

Your data

One row per borrower. Two columns are essential: the chance of default you predicted, and whether it defaulted.

Choose a file

LOAN_ACCT_NO,Opening PD,Default Flag,Risk Grade,Balance
L-00141,0.00318,N,A,3090219
L-00564,0.00938,N,BBB,3906868
L-00416,0.00945,N,BBB,2435569
L-00059,0.00302,N,A,2600291
L-00480,0.00868,N,BBB,764678
L-00505,0.00896,N,BBB,1969656
L-01082,0.05365,N,B,2819875
L-00757,0.01932,N,BB,2319685
L-00274,0.00288,N,A,1542857
L-01138,0.18357,Y,CCC,3020840
L-01060,0.05224,N,B,4119762
L-01005,0.05493,N,B,693421
L-00282,0.00311,N,A,1791045
L-00311,0.00897,N,BBB,2199382
L-01049,0.05614,N,B,2021780
L-00424,0.00921,N,BBB,2014786
L-00078,0.00295,N,A,2350794
L-00450,0.00882,N,BBB,4639014
L-00772,0.01913,N,BB,4822782
L-00232,0.00285,N,A,545357
L-00688,0.02105,N,BB,4798336
L-00874,0.01984,N,BB,4549261
L-00022,0.00299,N,A,4495718
L-00248,0.00309,N,A,2297659
L-00277,0.00299,N,A,459277
L-01003,0.05521,N,B,812449
L-01127,0.05177,N,B,2650662
L-00849,0.02019,N,BB,4058230
L-00346,0.00928,N,BBB,4261204
L-00776,0.01974,N,BB,1177274
L-01048,0.05343,N,B,733067
L-00679,0.01920,N,BB,3843329
L-01096,0.05757,N,B,4512319
L-00173,0.00300,N,A,3344838
L-00044,0.00293,N,A,1543075
L-00440,0.00950,N,BBB,1848398
L-00673,0.01901,N,BB,935332
L-00895,0.02047,N,BB,2428275
L-00755,0.01963,N,BB,4949778
L-00604,0.00869,N,BBB,1735166
L-00617,0.00937,N,BBB,3630993
L-00476,0.00864,N,BBB,2981834
L-00526,0.00864,N,BBB,1374560
L-00647,0.00900,N,BBB,4606083
L-01046,0.05751,N,B,356983
L-00780,0.01965,N,BB,3984100
L-00538,0.00873,N,BBB,2637723
L-00196,0.00291,N,A,3925415
L-00093,0.00312,N,A,4725360
L-00878,0.01910,N,BB,4600993
L-00970,0.05684,N,B,2327383
L-01052,0.05365,N,B,3123017
L-00833,0.02113,N,BB,3107721
L-01112,0.05313,N,B,584195
L-00106,0.00285,N,A,1665168
L-00429,0.00857,N,BBB,2367009
L-00033,0.00307,N,A,3037973
L-01035,0.05501,N,B,1202091
L-00309,0.00925,N,BBB,2106135
L-01184,0.18419,N,CCC,367418
L-00692,0.01972,N,BB,4043810
L-00969,0.05802,N,B,2791634
L-01038,0.05576,N,B,3764914
L-00524,0.00953,N,BBB,2867174
L-00529,0.00949,N,BBB,263207
L-00599,0.00926,N,BBB,1090457
L-00206,0.00285,N,A,965060
L-00036,0.00302,N,A,4817363
L-00334,0.00938,N,BBB,740444
L-00549,0.00852,N,BBB,4658749
L-00941,0.02010,N,BB,4816931
L-00917,0.01993,N,BB,1092845
L-00572,0.00923,N,BBB,1636319
L-00074,0.00289,N,A,800077
L-01126,0.05799,N,B,3721454
L-00591,0.00901,N,BBB,1717360
L-00363,0.00862,N,BBB,4723751
L-00735,0.02060,N,BB,3515868
L-00146,0.00290,N,A,1042346
L-00472,0.00874,N,BBB,4333177
L-00088,0.00293,N,A,2462266
L-00042,0.00296,N,A,3325945
L-00588,0.00863,N,BBB,4422076
L-00762,0.02083,N,BB,1992881
L-00430,0.00851,N,BBB,4935894
L-00483,0.00915,N,BBB,4268990
L-00851,0.01987,N,BB,865420
L-00556,0.00890,N,BBB,3728511
L-00222,0.00300,N,A,3872096
L-00243,0.00289,N,A,1775662
L-00765,0.02050,N,BB,4236178
L-00589,0.00911,N,BBB,3133235
L-00932,0.02100,N,BB,2497176
L-01057,0.05201,N,B,2943779
L-00977,0.05697,N,B,671641
L-00223,0.00284,N,A,1041647
L-00801,0.02077,N,BB,1451062
L-00291,0.00285,N,A,3444300
L-00441,0.00851,N,BBB,3926150
L-00650,0.00882,N,BBB,354329
L-00578,0.00901,N,BBB,1135155
L-00723,0.01955,N,BB,4312434
L-01045,0.05395,N,B,2050055
L-00854,0.02081,N,BB,3518324
L-00108,0.00310,N,A,287123
L-01019,0.05587,N,B,695435
L-00682,0.02107,N,BB,4546978
L-00984,0.05696,N,B,3010292
L-00190,0.00283,N,A,4069331
L-00496,0.00853,N,BBB,4144454
L-00455,0.00944,N,BBB,3714499
L-00154,0.00293,N,A,4282966
L-00500,0.00920,N,BBB,3181988
L-00419,0.00932,N,BBB,4166328
L-00985,0.05279,N,B,2848286
L-01001,0.05349,N,B,3307909
L-00859,0.01966,N,BB,1161122
L-00212,0.00289,N,A,2123682
L-00064,0.00300,N,A,1626475
L-00369,0.00909,N,BBB,3377482
L-00067,0.00311,N,A,2052978
L-00796,0.02035,N,BB,4355215
L-00107,0.00292,N,A,3846880
L-00171,0.00289,N,A,2285754
L-00150,0.00289,N,A,2929601
L-00276,0.00286,N,A,564124
L-00634,0.00924,N,BBB,4609250
L-00522,0.00917,N,BBB,1100261
L-00928,0.02031,N,BB,3444558
L-01156,0.17451,N,CCC,4898863
L-00468,0.00895,N,BBB,496145
L-00926,0.01904,N,BB,1451327
L-00769,0.01905,N,BB,2692359
L-01068,0.05314,N,B,964484
L-00165,0.00303,N,A,2847092
L-00616,0.00848,N,BBB,3092029
L-00733,0.02098,N,BB,4718085
L-00227,0.00298,N,A,3047590
L-00432,0.00933,N,BBB,2261161
L-00478,0.00927,N,BBB,2967766
L-00533,0.00916,N,BBB,2019708
L-00610,0.00923,N,BBB,3793505
L-00434,0.00892,N,BBB,2607150
L-01031,0.05540,N,B,4055926
L-01123,0.05644,N,B,1747822
L-00007,0.00295,N,A,1974039
L-00783,0.02059,N,BB,1955679
L-00743,0.02048,N,BB,499589
L-00888,0.02108,N,BB,3850751
L-00257,0.00316,N,A,4093078
L-01065,0.05431,N,B,272181
L-00642,0.00886,N,BBB,1232280
L-00246,0.00314,N,A,2074584
L-00343,0.00920,N,BBB,3664486
L-00530,0.00896,N,BBB,4830455
L-01176,0.17147,N,CCC,1561724
L-00347,0.00934,N,BBB,1813257
L-00218,0.00313,N,A,870565
L-00990,0.05473,N,B,694080
L-01148,0.18649,N,CCC,2440506
L-00445,0.00953,N,BBB,1800817
L-00497,0.00878,N,BBB,304945
L-00931,0.01974,N,BB,857756
L-00939,0.01890,N,BB,2579344
L-01093,0.05476,N,B,1416326
L-00439,0.00941,N,BBB,3342524
L-00839,0.02085,N,BB,2159345
L-01020,0.05353,N,B,1884688
L-00361,0.00904,N,BBB,3000887
L-00550,0.00942,N,BBB,468173
L-00400,0.00954,N,BBB,1867739
L-00614,0.00924,N,BBB,297329
L-00362,0.00895,N,BBB,1037401
L-01109,0.05407,N,B,3749464
L-01028,0.05532,N,B,3568586
L-01058,0.05420,N,B,1738486
L-00112,0.00292,N,A,4067603
L-00336,0.00879,N,BBB,3100438
L-00389,0.00885,N,BBB,2407808
L-00111,0.00316,N,A,2271073
L-00009,0.00300,N,A,1982693
L-00983,0.05688,N,B,2411243
L-00782,0.01943,N,BB,3513127
L-01160,0.18007,N,CCC,2036556
L-00079,0.00312,N,A,2753019
L-01089,0.05537,N,B,3236234
L-00054,0.00290,N,A,2569522
L-00652,0.02075,Y,BB,1836167
L-01197,0.17723,N,CCC,3494682
L-00275,0.00297,N,A,2532397
L-00236,0.00306,N,A,363527
L-00518,0.00920,N,BBB,1112981
L-00386,0.00940,N,BBB,4306403
L-00299,0.00309,N,A,2835517
L-00981,0.05421,N,B,2908656
L-00812,0.01993,N,BB,4152680
L-00195,0.00285,N,A,1107767
L-00523,0.00860,N,BBB,4301270
L-00470,0.00918,N,BBB,4982505
L-00493,0.00916,N,BBB,1121215
L-00600,0.00884,N,BBB,3478431
L-00615,0.00913,N,BBB,2208397
L-00010,0.00298,N,A,4246924
L-00481,0.00849,N,BBB,3845531
L-00184,0.00299,N,A,3500342
L-01102,0.05554,N,B,2401215
L-00174,0.00311,N,A,2485232
L-00674,0.02026,N,BB,4087955
L-00228,0.00293,N,A,4438865
L-00889,0.01928,N,BB,3297904
L-01141,0.17432,Y,CCC,810742
L-00625,0.00847,N,BBB,2371188
L-01117,0.05267,N,B,361487
L-00258,0.00312,N,A,1513120
L-00052,0.00292,N,A,1787170
L-00382,0.00918,N,BBB,4575177
L-01032,0.05813,N,B,3021221
L-00060,0.00318,N,A,4606042
L-00306,0.00850,N,BBB,3074309
L-00217,0.00308,N,A,4232799
L-00545,0.00866,N,BBB,4927522
L-00486,0.00846,N,BBB,813330
L-00546,0.00941,N,BBB,2808256
L-01119,0.05738,N,B,2511937
L-00244,0.00288,N,A,1076989
L-00982,0.05327,N,B,1975545
L-00885,0.01971,N,BB,3039091
L-01106,0.05641,N,B,3131442
L-00930,0.01951,N,BB,4443695
L-00879,0.02019,N,BB,2587403
L-00242,0.00306,N,A,1710410
L-00725,0.02114,N,BB,3274723
L-00163,0.00287,N,A,4358853
L-00965,0.05798,N,B,1527497
L-00730,0.01922,N,BB,266039
L-00262,0.00285,N,A,2088304
L-00501,0.00858,N,BBB,1666839
L-01149,0.17037,N,CCC,897116
L-00649,0.00934,N,BBB,1346108
L-00041,0.00313,N,A,1614613
L-00280,0.00305,N,A,3839664
L-00838,0.02077,N,BB,2080997
L-00936,0.02038,N,BB,733793
L-00333,0.00916,N,BBB,3347857
L-00260,0.00300,N,A,427671
L-00089,0.00304,N,A,768124
L-00528,0.00926,N,BBB,1227586
L-00211,0.00283,N,A,3813962
L-01153,0.17499,N,CCC,4109738
L-00284,0.00302,N,A,880953
L-00100,0.00292,N,A,536517
L-01077,0.05623,N,B,2401419
L-00558,0.00917,N,BBB,2150948
L-00748,0.01956,N,BB,3810102
L-00503,0.00893,N,BBB,542034
L-00329,0.00854,N,BBB,989872
L-00975,0.05710,N,B,882777
L-00256,0.00284,N,A,896783
L-01122,0.05649,N,B,4767650
L-00738,0.02035,N,BB,3283402
L-00407,0.00873,N,BBB,4992161
L-00325,0.00909,N,BBB,1819814
L-00721,0.01888,N,BB,1821879
L-00017,0.00312,N,A,3327325
L-00125,0.00298,N,A,4427768
L-00590,0.00924,N,BBB,4962064
L-00671,0.01933,N,BB,4218002
L-00710,0.02010,N,BB,1330423
L-00832,0.01913,N,BB,1373639
L-00267,0.00291,N,A,3609802
L-00620,0.00898,N,BBB,1408564
L-01192,0.17848,N,CCC,4193303
L-00391,0.00875,N,BBB,819986
L-00197,0.00293,N,A,459946
L-00027,0.00298,N,A,1831664
L-00913,0.01994,N,BB,3420342
L-00365,0.00934,N,BBB,2269691
L-01159,0.18115,N,CCC,1498894
L-00804,0.02063,N,BB,3144351
L-00966,0.05603,N,B,3306165
L-00385,0.00887,N,BBB,3772035
L-00532,0.00853,N,BBB,4212206
L-00685,0.02109,N,BB,3649374
L-00800,0.01938,N,BB,2158032
L-01110,0.05551,N,B,796033
L-00519,0.00908,N,BBB,3801238
L-00153,0.00318,N,A,482748
L-00148,0.00288,N,A,975777
L-00215,0.00306,N,A,4859742
L-00560,0.00886,N,BBB,2696382
L-00359,0.00914,N,BBB,358511
L-00456,0.00921,N,BBB,4914826
L-00438,0.00850,N,BBB,3677267
L-01098,0.05455,N,B,2254493
L-01108,0.05687,N,B,1209629
L-00919,0.02008,N,BB,4375504
L-00495,0.00937,N,BBB,4346604
L-00907,0.02102,N,BB,1201213
L-00300,0.00301,N,A,2066976
L-00571,0.00932,N,BBB,708180
L-00393,0.00874,N,BBB,3963508
L-00203,0.00283,N,A,1539754
L-01078,0.05324,N,B,4575288
L-00053,0.00285,N,A,4499447
L-01188,0.18690,N,CCC,4746482
L-00938,0.01987,N,BB,4600226
L-01087,0.05521,N,B,2381407
L-01059,0.05691,N,B,1153316
L-00562,0.00934,N,BBB,1492026
L-00192,0.00282,N,A,2940615
L-00758,0.01990,N,BB,3241560
L-00994,0.05561,N,B,4661100
L-00661,0.01926,Y,BB,4937046
L-00182,0.00284,N,A,4628890
L-00944,0.01957,N,BB,939943
L-01095,0.05423,N,B,1175438
L-00008,0.00316,N,A,764206
L-00348,0.00920,N,BBB,2991427
L-00512,0.00859,N,BBB,3586527
L-00719,0.02087,N,BB,2929049
L-00402,0.00887,N,BBB,727435
L-00172,0.00318,N,A,2998758
L-00968,0.05377,N,B,3315214
L-00395,0.00914,N,BBB,1420937
L-00098,0.00286,N,A,1358179
L-00120,0.00305,N,A,1357746
L-01024,0.05630,N,B,912121
L-00253,0.00297,N,A,1878885
L-01164,0.18859,N,CCC,1595288
L-00703,0.01950,N,BB,4457378
L-00752,0.01900,N,BB,1594455
L-00957,0.05268,Y,B,3763243
L-00492,0.00950,N,BBB,1814872
L-00047,0.00297,N,A,469973
L-00069,0.00295,N,A,388758
L-00553,0.00892,N,BBB,950355
L-00326,0.00892,N,BBB,1332517
L-01018,0.05731,N,B,3852385
L-00147,0.00284,N,A,2081332
L-00820,0.01999,N,BB,3667308
L-01076,0.05644,N,B,2561477
L-01118,0.05200,N,B,3343882
L-00876,0.01890,N,BB,1003861
L-00852,0.02027,N,BB,2182533
L-00117,0.00303,N,A,1761858
L-00744,0.02067,N,BB,1844046
L-01037,0.05434,N,B,2397896
L-00437,0.00936,N,BBB,4790743
L-00900,0.01888,N,BB,871351
L-00831,0.01886,N,BB,3263735
L-00249,0.00309,N,A,1166398
L-00491,0.00854,N,BBB,1670281
L-00927,0.02040,N,BB,1976398
L-00708,0.02074,N,BB,4671631
L-00861,0.02047,N,BB,659822
L-00958,0.05270,Y,B,782451
L-00841,0.02089,N,BB,503334
L-00214,0.00292,N,A,2728080
L-01064,0.05561,N,B,4645318
L-00204,0.00285,N,A,2486342
L-00066,0.00317,N,A,2876083
L-00561,0.00848,N,BBB,2829508
L-00081,0.00288,N,A,3967899
L-00425,0.00903,N,BBB,4680224
L-00123,0.00311,N,A,910894
L-00819,0.02012,N,BB,4724562
L-00886,0.01900,N,BB,767640
L-00475,0.00910,N,BBB,522011
L-01155,0.18073,N,CCC,3592426
L-01008,0.05488,N,B,367638
L-00375,0.00948,N,BBB,1878697
L-00986,0.05268,N,B,640443
L-00896,0.01901,N,BB,1251521
L-00698,0.01985,N,BB,2071636
L-00729,0.01976,N,BB,4649207
L-01074,0.05589,N,B,1239388
L-00471,0.00885,N,BBB,2704933
L-00189,0.00284,N,A,3541924
L-00370,0.00875,N,BBB,342623
L-00731,0.02073,N,BB,3461569
L-00515,0.00938,N,BBB,2128902
L-00514,0.00867,N,BBB,551332
L-00235,0.00302,N,A,3569076
L-00127,0.00310,N,A,2621691
L-00241,0.00311,N,A,2979422
L-01033,0.05823,N,B,2200595
L-00255,0.00307,N,A,4625687
L-00254,0.00296,N,A,1124262
L-00557,0.00950,N,BBB,1389730
L-00945,0.01956,N,BB,1908598
L-00504,0.00913,N,BBB,995065
L-00883,0.01905,N,BB,3990034
L-00032,0.00301,N,A,3871904
L-00788,0.02004,N,BB,3134642
L-00843,0.01962,N,BB,1116372
L-00423,0.00907,N,BBB,4169850
L-00310,0.00933,N,BBB,1336680
L-00013,0.00317,N,A,1589941
L-00104,0.00306,N,A,2151677
L-00967,0.05799,N,B,1939188
L-00290,0.00313,N,A,4873322
L-00129,0.00305,N,A,3504072
L-00039,0.00286,N,A,1510407
L-00714,0.02085,N,BB,4926730
L-00045,0.00304,N,A,376637
L-00996,0.05764,N,B,3525726
L-00576,0.00879,N,BBB,4340466
L-00443,0.00944,N,BBB,1500262
L-00324,0.00867,N,BBB,2674125
L-01158,0.17325,N,CCC,2452726
L-00791,0.02053,N,BB,836271
L-01157,0.16921,N,CCC,2827395
L-00272,0.00311,N,A,4694156
L-00822,0.01953,N,BB,2339563
L-00739,0.01929,N,BB,1942966
L-01051,0.05325,N,B,4149037
L-00640,0.00885,N,BBB,4073099
L-00824,0.01973,N,BB,3363654
L-00527,0.00930,N,BBB,2129957
L-01088,0.05333,N,B,3440931
L-01178,0.18316,N,CCC,3536623
L-00399,0.00849,N,BBB,4361177
L-00446,0.00871,N,BBB,3029755
L-00216,0.00293,N,A,559470
L-00827,0.01908,N,BB,2642161
L-00467,0.00872,N,BBB,3239254
L-01063,0.05685,N,B,1391825
L-00584,0.00924,N,BBB,3675468
L-01104,0.05597,N,B,1076717
L-00858,0.01980,N,BB,2326540
L-00628,0.00848,N,BBB,1757292
L-01150,0.18363,N,CCC,1384839
L-00999,0.05389,N,B,1611346
L-00070,0.00312,N,A,4993341
L-01187,0.17595,N,CCC,2491708
L-00012,0.00305,N,A,4343099
L-00305,0.00855,N,BBB,884738
L-01044,0.05818,N,B,4456431
L-00166,0.00303,N,A,1628412
L-00635,0.00952,N,BBB,325150
L-01042,0.05612,N,B,364453
L-00803,0.01893,N,BB,2495652
L-00910,0.01971,N,BB,4913930
L-00247,0.00295,N,A,853407
L-00263,0.00300,N,A,1302952
L-00778,0.02021,N,BB,1168440
L-00779,0.02052,N,BB,2707033
L-00356,0.00927,N,BBB,2446910
L-00071,0.00307,N,A,3939803
L-00633,0.00877,N,BBB,1386583
L-00048,0.00300,N,A,862144
L-01154,0.18759,N,CCC,2003773
L-00768,0.02079,N,BB,1216639
L-00364,0.00870,N,BBB,3544651
L-00051,0.00297,N,A,902428
L-01133,0.17776,Y,CCC,1924875
L-00751,0.02046,N,BB,334260
L-00043,0.00296,N,A,3613783
L-00947,0.02092,N,BB,259867
L-00119,0.00305,N,A,3106787
L-00914,0.02003,N,BB,4493916
L-00358,0.00932,N,BBB,3659394
L-00856,0.02013,N,BB,3707613
L-00916,0.02022,N,BB,378430
L-01084,0.05361,N,B,1435884
L-00213,0.00304,N,A,3885092
L-00374,0.00854,N,BBB,3194476
L-00605,0.00881,N,BBB,3069913
L-00924,0.02108,N,BB,1759998
L-00619,0.00891,N,BBB,3018041
L-00716,0.02035,N,BB,4794993
L-01101,0.05609,N,B,3250195
L-00159,0.00311,N,A,767921
L-00541,0.00910,N,BBB,3056108
L-00767,0.01984,N,BB,2715493
L-00020,0.00288,N,A,4065394
L-00199,0.00286,N,A,2528087
L-00627,0.00893,N,BBB,4317588
L-00638,0.00895,N,BBB,4161532
L-00149,0.00307,N,A,3597434
L-01145,0.18811,N,CCC,1791684
L-00678,0.02077,N,BB,1080094
L-00451,0.00854,N,BBB,490369
L-00156,0.00318,N,A,1001729
L-00327,0.00901,N,BBB,1166528
L-00575,0.00935,N,BBB,1006841
L-00415,0.00929,N,BBB,3023509
L-00388,0.00905,N,BBB,3725324
L-00598,0.00912,N,BBB,1756178
L-01137,0.17128,Y,CCC,1461443
L-01168,0.17729,N,CCC,2023904
L-00670,0.01896,N,BB,1761037
L-00806,0.01906,N,BB,3873641
L-00525,0.00936,N,BBB,442283
L-00194,0.00287,N,A,4618852
L-00942,0.01928,N,BB,317504
L-00704,0.01932,N,BB,4157475
L-00943,0.01965,N,BB,2607905
L-00828,0.01945,N,BB,596191
L-00458,0.00885,N,BBB,2485001
L-00328,0.00916,N,BBB,1864861
L-00770,0.02110,N,BB,3083624
L-00601,0.00896,N,BBB,742699
L-00339,0.00866,N,BBB,500309
L-00511,0.00871,N,BBB,3162075
L-00076,0.00317,N,A,2954185
L-01067,0.05458,N,B,442737
L-00687,0.01955,N,BB,699275
L-00315,0.00930,N,BBB,4652557
L-01086,0.05296,N,B,1917364
L-00608,0.00854,N,BBB,3803318
L-00773,0.01993,N,BB,1791193
L-00461,0.00856,N,BBB,694350
L-00898,0.02007,N,BB,2839737
L-00270,0.00311,N,A,4474214
L-00169,0.00293,N,A,1746728
L-01055,0.05199,N,B,309248
L-00909,0.02031,N,BB,3993061
L-00972,0.05615,N,B,4538582
L-00335,0.00927,N,BBB,4027456
L-00639,0.00950,N,BBB,2663218
L-00991,0.05516,N,B,489459
L-00281,0.00283,N,A,4188199
L-00929,0.01989,N,BB,834086
L-00320,0.00875,N,BBB,466411
L-00387,0.00921,N,BBB,1357660
L-00303,0.00854,Y,BBB,3472750
L-01170,0.17994,N,CCC,3825307
L-00866,0.01981,N,BB,1494475
L-00680,0.01939,N,BB,2382621
L-00278,0.00290,N,A,3953313
L-01091,0.05340,N,B,4393689
L-00086,0.00307,N,A,3778579
L-00689,0.01882,N,BB,2974905
L-00998,0.05543,N,B,3823453
L-00055,0.00309,N,A,4553800
L-00677,0.02067,N,BB,3337393
L-00136,0.00286,N,A,3219116
L-00890,0.01888,N,BB,2829135
L-00786,0.01993,N,BB,479321
L-00143,0.00307,N,A,3314905
L-01083,0.05323,N,B,3329498
L-00823,0.01889,N,BB,615852
L-01017,0.05441,N,B,4011049
L-00342,0.00889,N,BBB,2634597
L-00414,0.00880,N,BBB,2113654
L-00224,0.00290,N,A,3732519
L-00490,0.00881,N,BBB,1467879
L-01043,0.05505,N,B,1273010
L-00410,0.00875,N,BBB,747839
L-01061,0.05532,N,B,4878847
L-01200,0.17899,N,CCC,2560980
L-00586,0.00896,N,BBB,4484438
L-00114,0.00290,N,A,2578669
L-00105,0.00317,N,A,1773552
L-00477,0.00942,N,BBB,1355929
L-00144,0.00303,N,A,3472463
L-00830,0.01928,N,BB,4185609
L-00019,0.00282,N,A,4268802
L-00122,0.00306,N,A,913436
L-00955,0.05611,Y,B,3586972
L-00798,0.01914,N,BB,858631
L-00187,0.00294,N,A,1555692
L-00760,0.02060,N,BB,3761745
L-01174,0.18326,N,CCC,4071994
L-00643,0.00859,N,BBB,2978131
L-00231,0.00303,N,A,3331067
L-00447,0.00906,N,BBB,3302749
L-00624,0.00920,N,BBB,4039994
L-00401,0.00907,N,BBB,3897241
L-00681,0.02023,N,BB,485241
L-00314,0.00917,N,BBB,4342303
L-01136,0.18223,Y,CCC,4555110
L-01198,0.17170,N,CCC,2748452
L-00449,0.00850,N,BBB,320049
L-00229,0.00309,N,A,2309592
L-00816,0.01896,N,BB,551398
L-00566,0.00881,N,BBB,3635027
L-00185,0.00310,N,A,1303663
L-00954,0.05798,Y,B,2927737
L-00137,0.00300,N,A,343053
L-00865,0.01945,N,BB,2230451
L-00383,0.00864,N,BBB,352690
L-00205,0.00290,N,A,2484381
L-00618,0.00854,N,BBB,2790432
L-00971,0.05733,N,B,2850228
L-00239,0.00287,N,A,321289
L-00084,0.00315,N,A,4059864
L-00745,0.02103,N,BB,2683022
L-00313,0.00949,N,BBB,2819443
L-00221,0.00290,N,A,1293027
L-00049,0.00309,N,A,1088832
L-00494,0.00847,N,BBB,457363
L-01097,0.05185,N,B,4262659
L-00287,0.00296,N,A,3958040
L-00252,0.00313,N,A,848534
L-00091,0.00303,N,A,4870923
L-00469,0.00941,N,BBB,1225202
L-01128,0.05368,N,B,3090730
L-00807,0.01964,N,BB,378349
L-00261,0.00315,N,A,1331334
L-00442,0.00853,N,BBB,2557272
L-00684,0.02092,N,BB,4540579
L-00805,0.01895,N,BB,3527373
L-01023,0.05710,N,B,910939
L-00844,0.02031,N,BB,4339036
L-00269,0.00316,N,A,4784453
L-00028,0.00285,N,A,924492
L-00292,0.00318,N,A,3493152
L-00705,0.01959,N,BB,3533859
L-00115,0.00285,N,A,3681984
L-00380,0.00906,N,BBB,2740869
L-00789,0.02066,N,BB,1758301
L-00534,0.00888,N,BBB,707722
L-00542,0.00880,N,BBB,2674170
L-00090,0.00304,N,A,561453
L-00636,0.00923,N,BBB,958843
L-00240,0.00315,N,A,892633
L-01047,0.05683,N,B,4142627
L-00367,0.00927,N,BBB,2893342
L-00794,0.01885,N,BB,2154263
L-00595,0.00875,N,BBB,1640171
L-00405,0.00857,N,BBB,3396909
L-00397,0.00939,N,BBB,1032016
L-00694,0.01935,N,BB,2694334
L-00742,0.02003,N,BB,3595766
L-00162,0.00297,N,A,499339
L-00304,0.00919,N,BBB,2008298
L-00152,0.00306,N,A,2933020
L-00582,0.00874,N,BBB,4320069
L-00921,0.01962,N,BB,919204
L-00357,0.00886,N,BBB,4594658
L-00444,0.00936,N,BBB,2712879
L-00663,0.01947,Y,BB,3915552
L-00234,0.00314,N,A,4764622
L-00298,0.00288,N,A,3958447
L-01194,0.17593,N,CCC,503023
L-00581,0.00883,N,BBB,4843094
L-00554,0.00852,N,BBB,4002998
L-00462,0.00869,N,BBB,3161320
L-01054,0.05501,N,B,273499
L-00406,0.00857,N,BBB,4301228
L-01189,0.18808,N,CCC,4259090
L-00179,0.00296,N,A,4801838
L-00950,0.01908,N,BB,4055089
L-00829,0.02009,N,BB,662369
L-00881,0.02043,N,BB,4107508
L-00521,0.00863,N,BBB,2692352
L-00815,0.01937,N,BB,3771579
L-00952,0.05758,Y,B,4221085
L-00797,0.02065,N,BB,2567297
L-01050,0.05582,N,B,2715662
L-00031,0.00317,N,A,3843778
L-01002,0.05283,N,B,3204288
L-00718,0.02109,N,BB,2877051
L-00980,0.05599,N,B,282880
L-00850,0.01943,N,BB,2706774
L-01191,0.17254,N,CCC,4089030
L-00099,0.00290,N,A,364418
L-00993,0.05254,N,B,1306888
L-00612,0.00860,N,BBB,967593
L-00747,0.01936,N,BB,3126405
L-00792,0.02093,N,BB,4131616
L-00259,0.00303,N,A,1555031
L-00317,0.00848,N,BBB,2296073
L-01195,0.16969,N,CCC,3449546
L-00062,0.00310,N,A,3529038
L-00818,0.02083,N,BB,3255386
L-00517,0.00871,N,BBB,1062026
L-00283,0.00283,N,A,4132535
L-01094,0.05574,N,B,1740302
L-00037,0.00298,N,A,4870047
L-00177,0.00318,N,A,866614
L-00695,0.01904,N,BB,1066384
L-01169,0.18860,N,CCC,1209877
L-00330,0.00953,N,BBB,2194046
L-00579,0.00944,N,BBB,3786932
L-00892,0.02111,N,BB,1650388
L-00230,0.00301,N,A,2601146
L-00809,0.01905,N,BB,2394258
L-00436,0.00882,N,BBB,1772108
L-01012,0.05203,N,B,636921
L-00139,0.00315,N,A,2258491
L-00201,0.00316,N,A,3224567
L-00034,0.00292,N,A,4791844
L-01121,0.05812,N,B,289682
L-00349,0.00904,N,BBB,466759
L-00038,0.00298,N,A,4399243
L-00332,0.00893,N,BBB,4752126
L-00580,0.00890,N,BBB,1562858
L-00995,0.05593,N,B,3194518
L-00811,0.02066,N,BB,1249994
L-00488,0.00920,N,BBB,4281158
L-00848,0.02032,N,BB,4683318
L-00845,0.01906,N,BB,725681
L-01011,0.05682,N,B,547731
L-00392,0.00875,N,BBB,4612532
L-00498,0.00909,N,BBB,4709685
L-00307,0.00892,N,BBB,1617126
L-00911,0.01902,N,BB,2231371
L-00934,0.01956,N,BB,833188
L-00030,0.00291,N,A,496470
L-00279,0.00301,N,A,4770738
L-00857,0.02107,N,BB,4064611
L-00784,0.01911,N,BB,2943571
L-00046,0.00284,N,A,3780874
L-00657,0.02087,Y,BB,357797
L-01004,0.05316,N,B,3974216
L-00376,0.00906,N,BBB,4848640
L-00404,0.00906,N,BBB,4314808
L-00072,0.00291,N,A,709467
L-00004,0.00292,N,A,4278364
L-01124,0.05603,N,B,2089832
L-00188,0.00283,N,A,3307116
L-00676,0.01894,N,BB,4576911
L-00962,0.05240,N,B,3880325
L-00665,0.02056,Y,BB,3498552
L-00245,0.00314,N,A,4832780
L-00836,0.01995,N,BB,2702507
L-01099,0.05340,N,B,895471
L-00997,0.05611,N,B,4400985
L-00167,0.00290,N,A,3212618
L-01179,0.17604,N,CCC,1217875
L-00736,0.01946,N,BB,752831
L-00109,0.00289,N,A,2550848
L-00002,0.00296,N,A,3593987
L-00764,0.01951,N,BB,253232
L-00360,0.00871,N,BBB,3927059
L-00431,0.00932,N,BBB,4409235
L-00133,0.00311,N,A,3473228
L-00923,0.01991,N,BB,459943
L-00482,0.00852,N,BBB,1540409
L-00607,0.00867,N,BBB,846005
L-00691,0.02024,N,BB,2402011
L-00613,0.00938,N,BBB,2862865
L-00390,0.00856,N,BBB,398502
L-00814,0.02041,N,BB,732378
L-00675,0.02053,N,BB,2207629
L-00025,0.00293,N,A,3419006
L-00537,0.00882,N,BBB,4211726
L-00606,0.00934,N,BBB,4387161
L-01015,0.05722,N,B,1263344
L-01193,0.17567,N,CCC,3959839
L-00862,0.02024,N,BB,4140942
L-01171,0.18550,N,CCC,4756317
L-01100,0.05396,N,B,2128005
L-00959,0.05226,Y,B,1285286
L-00543,0.00849,N,BBB,1823705
L-00191,0.00295,N,A,4946399
L-00318,0.00953,N,BBB,4991445
L-00880,0.02079,N,BB,3042212
L-00473,0.00855,N,BBB,2735711
L-00302,0.00949,Y,BBB,792114
L-00412,0.00881,N,BBB,2046539
L-00340,0.00865,N,BBB,3607650
L-00753,0.02067,N,BB,3935583
L-00696,0.02025,N,BB,1296979
L-00531,0.00880,N,BBB,3037630
L-00050,0.00283,N,A,877234
L-00826,0.02005,N,BB,1589782
L-00018,0.00283,N,A,2491526
L-00377,0.00914,N,BBB,3182720
L-00835,0.02012,N,BB,3637460
L-00026,0.00290,N,A,1989228
L-01165,0.18059,N,CCC,4262311
L-00138,0.00298,N,A,2604568
L-00937,0.01885,N,BB,1036779
L-00882,0.01996,N,BB,563709
L-00555,0.00887,N,BBB,1255921
L-01000,0.05737,N,B,359022
L-00877,0.02102,N,BB,1956704
L-00289,0.00305,N,A,1498159
L-00871,0.01940,N,BB,2180781
L-01090,0.05300,N,B,1495546
L-00920,0.01970,N,BB,3667823
L-00872,0.02009,N,BB,2021585
L-00720,0.01984,N,BB,1676171
L-00341,0.00878,N,BBB,3302408
L-00384,0.00904,N,BBB,3022546
L-00408,0.00863,N,BBB,4353471
L-00979,0.05651,N,B,4658503
L-00655,0.01959,Y,BB,4791666
L-00799,0.01980,N,BB,4565775
L-00095,0.00318,N,A,2060688
L-00454,0.00890,N,BBB,4815558
L-00953,0.05683,Y,B,699751
L-00592,0.00920,N,BBB,4566225
L-00840,0.02112,N,BB,3766287
L-01073,0.05678,N,B,3665644
L-00082,0.00312,N,A,1912873
L-00667,0.02022,Y,BB,533493
L-00452,0.00925,N,BBB,1391815
L-00118,0.00316,N,A,3470302
L-00577,0.00850,N,BBB,2307760
L-00040,0.00287,N,A,1986478
L-00132,0.00292,N,A,1106809
L-01175,0.18034,N,CCC,3997016
L-00567,0.00926,N,BBB,1678399
L-00630,0.00877,N,BBB,4668588
L-01130,0.05192,N,B,3938889
L-00781,0.01917,N,BB,438431
L-01069,0.05349,N,B,1943595
L-01173,0.17881,N,CCC,3739305
L-00658,0.02114,Y,BB,1086292
L-00058,0.00303,N,A,3574917
L-00322,0.00924,N,BBB,4711704
L-01081,0.05406,N,B,651719
L-00183,0.00298,N,A,3889599
L-00732,0.01974,N,BB,735796
L-00905,0.02041,N,BB,4365855
L-00949,0.01943,N,BB,1623016
L-00734,0.02097,N,BB,2700094
L-00350,0.00865,N,BBB,1707537
L-00186,0.00294,N,A,2009643
L-01146,0.18778,N,CCC,1797006
L-01116,0.05398,N,B,4565232
L-00891,0.02110,N,BB,2615438
L-00413,0.00923,N,BBB,4611953
L-00540,0.00942,N,BBB,1232861
L-00001,0.00296,Y,A,2895641
L-00352,0.00858,N,BBB,4402537
L-00016,0.00311,N,A,1468284
L-00574,0.00878,N,BBB,4418636
L-00398,0.00851,N,BBB,2418047
L-00068,0.00294,N,A,3661308
L-00077,0.00309,N,A,3744547
L-00202,0.00310,N,A,2127196
L-00485,0.00887,N,BBB,3492624
L-00539,0.00886,N,BBB,1327387
L-00312,0.00951,N,BBB,1497742
L-00860,0.01944,N,BB,1468100
L-00901,0.02111,N,BB,2084892
L-00645,0.00909,N,BBB,1153363
L-00666,0.01955,Y,BB,4195073
L-00175,0.00293,N,A,1862788
L-01027,0.05304,N,B,1837496
L-00134,0.00286,N,A,4698014
L-01079,0.05747,N,B,1386740
L-00870,0.02105,N,BB,2693016
L-00750,0.01884,N,BB,4557363
L-00787,0.01923,N,BB,3409501
L-00668,0.02075,Y,BB,4674295
L-00940,0.02055,N,BB,4049190
L-00766,0.01899,N,BB,3265722
L-00265,0.00284,N,A,1300200
L-00113,0.00315,N,A,4751391
L-00653,0.02098,Y,BB,2124629
L-01114,0.05786,N,B,1442123
L-00513,0.00847,N,BBB,1650317
L-01177,0.18590,N,CCC,3531690
L-00904,0.01990,N,BB,1646252
L-00637,0.00895,N,BBB,2595181
L-00741,0.02084,N,BB,1814755
L-00551,0.00946,N,BBB,1643621
L-00774,0.02095,N,BB,1442284
L-00155,0.00311,N,A,3175039
L-00659,0.02113,Y,BB,1310548
L-00974,0.05711,N,B,2487251
L-00219,0.00300,N,A,1382686
L-00296,0.00312,N,A,1519932
L-00976,0.05731,N,B,822042
L-01166,0.18085,N,CCC,1550452
L-00092,0.00288,N,A,2861886
L-00868,0.02105,N,BB,3223567
L-00180,0.00286,N,A,4571634
L-00884,0.02033,N,BB,1375384
L-00644,0.00878,N,BBB,4438505
L-01167,0.18675,N,CCC,3805293
L-00085,0.00309,N,A,3863879
L-01125,0.05575,N,B,1504075
L-00973,0.05318,N,B,3597992
L-00157,0.00283,N,A,549647
L-00548,0.00871,N,BBB,560481
L-00516,0.00949,N,BBB,2984144
L-00520,0.00885,N,BBB,1769789
L-00080,0.00317,N,A,2660957
L-00626,0.00866,N,BBB,2045970
L-00863,0.02082,N,BB,3693894
L-00855,0.02094,N,BB,1834237
L-00096,0.00282,N,A,2020250
L-00116,0.00313,N,A,4078543
L-00198,0.00309,N,A,1886569
L-00285,0.00314,N,A,312924
L-00722,0.01895,N,BB,2969759
L-00023,0.00297,N,A,2511561
L-00506,0.00858,N,BBB,4443864
L-01183,0.18231,N,CCC,3423698
L-00686,0.01963,N,BB,3421651
L-01147,0.18756,N,CCC,1877833
L-00918,0.01959,N,BB,4872425
L-00208,0.00302,N,A,525847
L-00003,0.00291,N,A,3668111
L-00421,0.00934,N,BBB,3632068
L-00837,0.01913,N,BB,2333598
L-00178,0.00285,N,A,1409129
L-01071,0.05649,N,B,1067630
L-00935,0.02008,N,BB,4608056
L-00887,0.01892,N,BB,1954072
L-00702,0.02065,N,BB,1142462
L-00946,0.02027,N,BB,4352619
L-00128,0.00287,N,A,474862
L-01105,0.05666,N,B,271487
L-00802,0.02094,N,BB,971643
L-00749,0.02033,N,BB,1855347
L-01092,0.05765,N,B,3718071
L-00161,0.00313,N,A,607856
L-00656,0.02024,Y,BB,4258071
L-00479,0.00856,N,BBB,1324931
L-00083,0.00307,N,A,4570836
L-00087,0.00291,N,A,2431326
L-00014,0.00307,N,A,937088
L-00266,0.00310,N,A,1334756
L-00609,0.00877,N,BBB,2609148
L-00130,0.00301,N,A,4937106
L-00094,0.00289,N,A,3227070
L-00775,0.01903,N,BB,2343896
L-00867,0.01917,N,BB,1379358
L-00711,0.01885,N,BB,2726532
L-00250,0.00287,N,A,4080332
L-00961,0.05187,Y,B,4888721
L-01036,0.05719,N,B,4754285
L-01190,0.18502,N,CCC,2187782
L-00029,0.00286,N,A,1462746
L-00596,0.00908,N,BBB,3773540
L-00771,0.02014,N,BB,3014724
L-00897,0.02046,N,BB,4074868
L-01139,0.17838,Y,CCC,3232244
L-00403,0.00863,N,BBB,4731371
L-00906,0.02115,N,BB,276969
L-00345,0.00866,N,BBB,2018870
L-00268,0.00291,N,A,3778507
L-00701,0.01924,N,BB,4249590
L-00583,0.00889,N,BBB,4531375
L-00573,0.00848,N,BBB,1165531
L-01181,0.19052,N,CCC,4531904
L-00323,0.00935,N,BBB,2467105
L-00536,0.00937,N,BBB,1208815
L-01041,0.05371,N,B,4607048
L-00690,0.02064,N,BB,1213313
L-00756,0.01915,N,BB,3826966
L-00460,0.00912,N,BBB,1735913
L-00641,0.00943,N,BBB,1587300
L-00459,0.00906,N,BBB,1689368
L-00544,0.00948,N,BBB,762912
L-01115,0.05613,N,B,3566652
L-00435,0.00911,N,BBB,4707668
L-00508,0.00917,N,BBB,2282847
L-00464,0.00936,N,BBB,2369106
L-01034,0.05218,N,B,3914292
L-00597,0.00891,N,BBB,1512849
L-00717,0.01942,N,BB,273887
L-00864,0.01971,N,BB,2597592
L-00209,0.00293,N,A,1959575
L-00142,0.00314,N,A,1308686
L-00193,0.00305,N,A,2372292
L-00825,0.01981,N,BB,635585
L-00073,0.00297,N,A,485736
L-00396,0.00881,N,BBB,842746
L-00295,0.00282,N,A,2812640
L-00988,0.05432,N,B,666454
L-00124,0.00317,N,A,4621467
L-00706,0.02078,N,BB,434504
L-00418,0.00862,N,BBB,398170
L-01056,0.05341,N,B,2352895
L-00925,0.01925,N,BB,1853081
L-00065,0.00287,N,A,3443495
L-00273,0.00304,N,A,2650522
L-01132,0.17815,Y,CCC,4705542
L-00176,0.00297,N,A,453417
L-00672,0.02008,N,BB,1758942
L-01080,0.05620,N,B,603647
L-00024,0.00309,N,A,4899163
L-00509,0.00886,N,BBB,1853217
L-00817,0.01900,N,BB,4548144
L-00417,0.00853,N,BBB,4818255
L-00353,0.00902,N,BBB,3800772
L-00715,0.02039,N,BB,1334526
L-00487,0.00949,N,BBB,3447829
L-00140,0.00289,N,A,690083
L-00409,0.00895,N,BBB,4144733
L-00629,0.00862,N,BBB,3148727
L-00875,0.02020,N,BB,1104750
L-00237,0.00283,N,A,866027
L-01111,0.05595,N,B,3520827
L-01151,0.18776,N,CCC,2022861
L-01062,0.05172,N,B,2943110
L-00006,0.00304,N,A,1993254
L-00761,0.01923,N,BB,542874
L-00354,0.00857,N,BBB,4539740
L-00559,0.00902,N,BBB,3781619
L-00899,0.01936,N,BB,1348053
L-00699,0.02003,N,BB,1385688
L-00552,0.00915,N,BBB,4974223
L-00466,0.00944,N,BBB,1774492
L-01053,0.05767,N,B,1221492
L-00808,0.01895,N,BB,474352
L-00631,0.00945,N,BBB,702792
L-00207,0.00290,N,A,4580492
L-00057,0.00297,N,A,269434
L-00847,0.02027,N,BB,2947729
L-00170,0.00284,N,A,1489128
L-00960,0.05582,Y,B,2223333
L-01085,0.05221,N,B,3161588
L-01185,0.18349,N,CCC,2139517
L-01040,0.05469,N,B,2717112
L-01144,0.17832,N,CCC,4111969
L-00337,0.00931,N,BBB,3531436
L-00321,0.00874,N,BBB,3211351
L-00568,0.00952,N,BBB,1286686
L-01029,0.05247,N,B,1143834
L-00654,0.02051,Y,BB,2828129
L-00319,0.00892,N,BBB,2735727
L-00908,0.01921,N,BB,1295348
L-00724,0.01906,N,BB,674211
L-00790,0.01986,N,BB,1883945
L-00075,0.00302,N,A,1127339
L-00547,0.00881,N,BBB,884313
L-00948,0.02061,N,BB,4261739
L-01135,0.18194,Y,CCC,1659196
L-01013,0.05630,N,B,4416318
L-00422,0.00926,N,BBB,553503
L-01180,0.17440,N,CCC,428647
L-00015,0.00317,N,A,1818662
L-00737,0.01963,N,BB,3950996
L-01010,0.05373,N,B,4713913
L-00623,0.00896,N,BBB,3790452
L-00338,0.00934,N,BBB,2097490
L-00484,0.00864,N,BBB,795028
L-00220,0.00290,N,A,279570
L-01182,0.18888,N,CCC,3559524
L-01075,0.05607,N,B,2367300
L-00697,0.01933,N,BB,1349847
L-00293,0.00283,N,A,1147974
L-00712,0.01950,N,BB,4381164
L-00158,0.00294,N,A,4884180
L-00297,0.00312,N,A,3065473
L-01162,0.17845,N,CCC,4064783
L-00759,0.01983,N,BB,1470346
L-00160,0.00291,N,A,3099440
L-01131,0.17221,Y,CCC,1358878
L-00011,0.00303,N,A,3007628
L-00233,0.00289,N,A,3204061
L-01016,0.05601,N,B,3472701
L-00922,0.02105,N,BB,2749667
L-00912,0.02094,N,BB,3518460
L-00669,0.02023,N,BB,4254764
L-01186,0.18444,N,CCC,3840390
L-00603,0.00944,N,BBB,1368323
L-01006,0.05643,N,B,536208
L-00101,0.00299,N,A,2278051
L-00061,0.00284,N,A,1678995
L-00709,0.02109,N,BB,4725256
L-00587,0.00927,N,BBB,4395316
L-00660,0.01926,Y,BB,1740693
L-00056,0.00313,N,A,1621824
L-00693,0.02102,N,BB,3969361
L-00502,0.00848,N,BBB,3564352
L-00869,0.01984,N,BB,4207670
L-00457,0.00940,N,BBB,2391027
L-00168,0.00300,N,A,2005288
L-00371,0.00946,N,BBB,1839904
L-01129,0.05601,N,B,3099517
L-00131,0.00298,N,A,3611640
L-00063,0.00294,N,A,1325391
L-00110,0.00284,N,A,4759886
L-00964,0.05823,N,B,1889862
L-00316,0.00859,N,BBB,2181270
L-00121,0.00309,N,A,1561315
L-00331,0.00953,N,BBB,3607306
L-00126,0.00286,N,A,1172355
L-00308,0.00875,N,BBB,905155
L-00795,0.02104,N,BB,541221
L-00585,0.00888,N,BBB,1787405
L-00813,0.01910,N,BB,3505316
L-00271,0.00283,N,A,2875952
L-01107,0.05317,N,B,551058
L-00005,0.00318,N,A,4974115
L-01103,0.05530,N,B,593566
L-01025,0.05650,N,B,4844202
L-00763,0.02063,N,BB,3778468
L-01009,0.05674,N,B,1996336
L-00200,0.00283,N,A,2541200
L-00366,0.00951,N,BBB,2932243
L-00510,0.00900,N,BBB,3509476
L-00740,0.01928,N,BB,4448981
L-00632,0.00900,N,BBB,1721308
L-01140,0.18978,Y,CCC,3095954
L-00646,0.00893,N,BBB,966117
L-00448,0.00937,N,BBB,438236
L-00465,0.00921,N,BBB,3042857
L-00956,0.05214,Y,B,3601372
L-00379,0.00893,N,BBB,2131969
L-00834,0.02081,N,BB,4021436
L-00727,0.02095,N,BB,2680433
L-00894,0.02108,N,BB,2306419
L-00368,0.00917,N,BBB,423458
L-01113,0.05251,N,B,4859607
L-00842,0.01906,N,BB,546254
L-01152,0.17398,N,CCC,1682812
L-00499,0.00881,N,BBB,2984088
L-00707,0.01970,N,BB,1966171
L-00426,0.00880,N,BBB,2718279
L-00648,0.00869,N,BBB,4879911
L-00963,0.05493,N,B,3160175
L-01070,0.05496,N,B,3661403
L-00777,0.01953,N,BB,3784151
L-00903,0.01983,N,BB,874052
L-01014,0.05650,N,B,3481206
L-00893,0.01924,N,BB,1135583
L-01021,0.05793,N,B,1902771
L-00489,0.00853,N,BBB,273406
L-01039,0.05451,N,B,4728118
L-00035,0.00313,N,A,3050612
L-00754,0.02031,N,BB,3335853
L-00351,0.00890,N,BBB,1093270
L-00621,0.00907,N,BBB,1279976
L-00286,0.00300,N,A,3345116
L-00373,0.00851,N,BBB,3514649
L-01199,0.17820,N,CCC,3010127
L-00987,0.05583,N,B,3076358
L-00570,0.00861,N,BBB,2888924
L-00420,0.00926,N,BBB,3860687
L-00951,0.05570,Y,B,4186179
L-00593,0.00856,N,BBB,4353213
L-00288,0.00313,N,A,542536
L-00474,0.00944,N,BBB,2717747
L-00535,0.00861,N,BBB,253974
L-00683,0.02018,N,BB,2095311
L-00344,0.00857,N,BBB,2271649
L-00651,0.02073,Y,BB,1860178
L-00394,0.00953,N,BBB,1921862
L-01120,0.05519,N,B,3160360
L-00427,0.00860,N,BBB,4372448
L-00102,0.00306,N,A,904210
L-00411,0.00868,N,BBB,3813525
L-00569,0.00859,N,BBB,3733160
L-00463,0.00900,N,BBB,4953287
L-00664,0.01935,Y,BB,640072
L-00226,0.00293,N,A,1807121
L-00978,0.05363,N,B,682543
L-00989,0.05668,N,B,4254497
L-01163,0.18007,N,CCC,1819042
L-00793,0.02052,N,BB,1114586
L-00565,0.00941,N,BBB,4125360
L-00746,0.01941,N,BB,1739434
L-00225,0.00307,N,A,3365494
L-00873,0.01972,N,BB,1288869
L-00713,0.02049,N,BB,3038383
L-00378,0.00894,N,BBB,4762350
L-00726,0.02076,N,BB,3939993
L-00915,0.01881,N,BB,3693863
L-00151,0.00316,N,A,480329
L-00933,0.02112,N,BB,4338066
L-00301,0.00878,Y,BBB,2721837
L-00846,0.01992,N,BB,3140654
L-00238,0.00288,N,A,4861209
L-00821,0.01887,N,BB,3432121
L-00294,0.00312,N,A,4601707
L-00210,0.00293,N,A,4221742
L-00853,0.02025,N,BB,870201
L-01142,0.18980,Y,CCC,3366631
L-00902,0.02032,N,BB,4025859
L-00355,0.00944,N,BBB,3062656
L-01066,0.05287,N,B,2328243
L-00700,0.01962,N,BB,435965
L-00785,0.01964,N,BB,3290383
L-00433,0.00853,N,BBB,252577
L-00728,0.02102,N,BB,1191785
L-00372,0.00910,N,BBB,1866679
L-00662,0.02036,Y,BB,4157170
L-00164,0.00299,N,A,2723502
L-00507,0.00907,N,BBB,2646108
L-01007,0.05296,N,B,4909257
L-01030,0.05308,N,B,701214
L-00611,0.00924,N,BBB,1025969
L-00453,0.00872,N,BBB,3577739
L-00021,0.00308,N,A,4202007
L-00594,0.00860,N,BBB,4737123
L-00181,0.00300,N,A,1712161
L-01026,0.05591,N,B,1984995
L-00622,0.00936,N,BBB,3670945
L-00135,0.00290,N,A,1390762
L-01022,0.05583,N,B,2768000
L-00264,0.00295,N,A,2423766
L-00810,0.02014,N,BB,1084596
L-00381,0.00886,N,BBB,2900909
L-00145,0.00284,N,A,2163675
L-01072,0.05342,N,B,2698345
L-00097,0.00299,N,A,3430323
L-00251,0.00301,N,A,4423356
L-01161,0.17607,N,CCC,1147747
L-00428,0.00897,N,BBB,4982101
L-01143,0.18760,N,CCC,3257537
L-01196,0.18009,N,CCC,1132936
L-00103,0.00310,N,A,3756122
L-01134,0.17684,Y,CCC,3347385
L-01172,0.17459,N,CCC,2603020
L-00563,0.00951,N,BBB,3682838
L-00992,0.05791,N,B,1321977
L-00602,0.00939,N,BBB,2474567

This is an example book of 1,200 synthetic borrowers, not your data. It is here so the page shows what it does before you have fetched anything. Replace it when you are ready.

Read 1,200 rows and 5 columns.

Which column is which

Corrected automatically where the heading is recognisable. Change anything that is wrong.

Exposure or borrower reference

Whatever identifies the account. Used only to report which rows failed.

Predicted chance of default, at the start of the period *

The probability of default (PD) the model gave BEFORE the outcome was known. A figure refreshed during the period tests hindsight, not prediction.

Did it default? *

What actually happened by the end of the period.

Rating grade at the start

Lets the test run grade by grade. Without it only a total for the whole book is possible, and a total can hide one grade set too low behind another set too high.

Exposure amount

Used to weight the average predicted chance of default. Rows count equally without it.

The predicted chance of default is being read as decimals

Every value sits in the decimal range. Read the wrong way round, a model would look a hundred times more conservative than it is and pass every test on this page.

Can it tell good from bad?

Whether the borrowers it called riskier are the ones that defaulted. A model that cannot rank risk cannot be fixed by adjusting its rates.

Passed

It ranks risk well: the borrowers it called riskier are the ones that defaulted (Gini 0.568, 95% interval 0.408 to 0.729, on 45 defaults).

Defaults observed

Default rate

3.75%

The statistics, for a model validator

Gini

0.568

AUC

0.7842

AUC, 95% interval

0.704 – 0.865

0.522

Gini runs from 0, no better than chance, to 1, a perfect ranking; 0.20 is the usual floor. AUC is the same measure on a scale from 0.5. KS is the widest gap between how defaulted and performing borrowers were scored.

Are the levels right?

Whether the predicted chance of default matched what happened, grade by grade — because a total can hide one grade set too low behind another set too high.

Failed

The total for the book looks acceptable, but 1 grade (BB) is set too low — more defaults than a correct rate would give once in a hundred. A total that hides one grade too low behind another too high is not a model that is right.

Grade	Borrowers	Defaults	Predicted	Observed	Jeffreys p
A	300	1	0.30%	0.33%	0.3847	Passed
BBB	350	3	0.90%	0.86%	0.4931	Passed
BB	300	18	2.00%	6.00%	<0.0001	Failed
B	180	11	5.51%	6.11%	0.3453	Passed
CCC	70	12	18.08%	17.14%	0.5680	Passed

The last column is how often a grade priced correctly would see at least this many defaults (the Jeffreys test). Below 1% fails; below 5% is worth explaining.

The statistics, for a model validator

Hosmer–Lemeshow p

0.0002

Spiegelhalter p

0.0084

Brier score

0.03473

— calibration error

0.00041

— resolution

0.00179

Tests of the whole book at once. Each asks the same question as the table — do predictions and outcomes agree — in a different way, and none of them can see a grade the others net away.

How much defaults move together

Allows for defaults arriving in the same bad year, rather than one at a time.

0.15Set this to the asset correlation your rates were built with — 0.15 means 15%. Higher means a bad year is less surprising, so the test is more forgiving, which is why it must match the model rather than be chosen here.