I have a dataframe with columns of data where most is populated but some entries are NaN.
I have many columns of data values with a collection of descriptive columns as well. Data sample looks like this:
name trade_date clean_pub_date day_lag ticker transaction_type asset_type clean_amt 1m_p_ret 3m_p_ret 6m_p_ret 12m_p_ret 1m_t_ret 3m_t_ret 6m_t_ret 12m_t_retHon. Kevin Hern 2023-03-27 2023-03-28 1 WMB P ST 7500 0.0325596327213624 0.0916310651892829 0.209011889793836 0.407737489158621 0.0176287235543249 0.0893671114363008 0.206504497788805 0.388415020898048Hon. Jared Moskowitz 2023-03-27 2023-03-31 4 ADI P ST 7500 -0.0744347159107126 -0.0346437023298912 -0.115388412186706 0.0160020651692088 -0.0222439467606298 0.0133274320240624 -0.0414668810057205 0.042212539878232Hon. Jared Moskowitz 2023-03-27 2023-03-31 4 ADI P ST 7500 -0.0744347159107126 -0.0346437023298912 -0.115388412186706 0.0160020651692088 -0.0222439467606298 0.0133274320240624 -0.0414668810057205 0.042212539878232Hon. Jared Moskowitz 2023-03-27 2023-03-31 4 ADI P ST 7500 -0.0744347159107126 -0.0346437023298912 -0.115388412186706 0.0160020651692088 -0.0222439467606298 0.0133274320240624 -0.0414668810057205 0.042212539878232Hon. Josh Gottheimer 2023-03-09 2023-04-06 28 TSM P ST 7500 -0.0520834167337098 0.123814025123852 -0.0446267558738616 0.595117795767867 0.00678891463471643 0.134249211956264 0.0743782722398085 0.678835934021172Hon. Josh Gottheimer 2023-03-14 2023-04-06 23 APPF P ST 7500 0.0982549180838661 0.376045865131344 0.443382006265114 0.851382626392817 0.08007380674002 0.458955919210129 0.594846361257788 0.895417227472634Hon. Michael K. Simpson 2023-03-14 2023-03-17 3 USB P ST 32500 0.0946277179229686 0.0238876479022689 0.0973928153829178 0.360966257218354 -0.0441170992422953 -0.13111379775811 -0.00721832444817516 0.223666620799144Hon. Daniel Goldman 2023-01-13 2023-02-09 27 AJINY P ST 7500 0.0627803022078508 0.140935288766908 0.266175517255304 0.228058968476267 -0.0173298783961181 0.152759895534099 0.250962764220735 0.276315810414442Hon. Daniel Goldman 2023-01-13 2023-02-09 27 BAYRY P ST 7500 -0.0748926764291779 0.024757557892245 -0.0825067088504125 -0.519225862888112 0.0281870043179597 0.0839205732235453 -0.0771860668183186 -0.39034316672245Hon. Daniel Goldman 2023-01-13 2023-02-09 27 DNTUY P ST 7500 0.0678599859331164 0.137698235937564 0.0271628133072503 -0.148688924638075 -0.00439564506500045 0.133124010057626 0.0455886260460698 -0.162951349196126Hon. Daniel Goldman 2023-01-13 2023-02-09 27 DEO P ST 32500 0.000611121271014925 0.0728358519103816 0.0007867613724446 -0.129753323235326 -0.0483052090874593 0.0349147772678002 -0.0456513467069262 -0.207635916805358Hon. Daniel Goldman 2023-01-13 2023-02-09 27 FANUY P ST 7500 0.0104469129401044 0 -0.164248403882357 -0.230992434589453 0.0672839568675496 0.0672839568675496 0.0512345607788387 -0.0672840157363319Hon. Daniel Goldman 2023-01-13 2023-02-09 27 HLN P ST 7500 -0.0605685251224887 0.0921340494800955 0.0473029752043332 0.0180360261060983 0.0225282389561827 0.103280892559871 0.0351927677715116 0.0764206766478224Hon. Daniel Goldman 2023-01-13 2023-02-09 27 KDDIY P ST 7500 0.00867245606576961 0.0460307234551844 -0.0473649132080014 -0.00333556981309635 -0.00528750317917703 0.00594845683457668 0.00727030111336591 0.115003351727124Hon. Daniel Goldman 2023-01-13 2023-02-09 27 KNBWY P ST 7500 -0.0135048254386579 0.0102893791285894 -0.0855305406683214 -0.0726688167575846 0.0497311670313767 0.0658601837835502 -0.00268816945869565 -0.0282258113618117Hon. Daniel Goldman 2023-01-13 2023-02-09 27 MSADY P ST 7500 -0.00125147381631208 0.0713391996243071 0.123279212312712 0.258448142031357 0.0142063311841789 -0.00926501396493784 0.13650410907584 0.231006845072687Hon. Daniel Goldman 2023-01-13 2023-02-09 27 RHHBY P ST 32500 -0.0526454161461736 0.0690128140720261 0.0263064779327058 -0.126679953884905 -0.0375349124577606 0.0260710017123225 0.00739600734214108 -0.0452094559194888Hon. Daniel Goldman 2023-01-13 2023-02-09 27 SVNDY P ST 7500 0.0129521660364387 0.0375167113954387 -0.0759268056053813 -0.0794998257382803 -0.0352793216407492 -0.0839983018909187 -0.0541788960080017 -0.176816449212095Hon. Daniel Goldman 2023-01-13 2023-02-09 27 SSDOY P ST 7500 -0.0938563603529217 -0.0204122560105336 -0.0892536079691367 -0.431659016915056 -0.0211081978052522 0.0133955721794217 -0.0694134757837956 -0.419728031520997Hon. Daniel Goldman 2023-01-13 2023-02-09 27 SOMMY P ST 7500 -0.0109132990076309 -0.0379092391267433 -0.191843778395228 -0.419873608846066 -0.0295890912617722 -0.0421918059048587 -0.149041110522127 -0.339726016946035Hon. Daniel Goldman 2023-01-13 2023-02-09 27 STBFY P ST 7500 0.0778834417188183 0.111771765887353 0.0514268600958618 -0.0154578022353854 0.0234656660622905 0.12996388317341 0.0734054904783485 0.0171480043935579Hon. Daniel Goldman 2023-01-13 2023-02-09 27 TRYIY P ST 7500 -0.0207074864502548 -0.0232959839697141 -0.0258843992049368 -0.220017269815608 0.0311388246082596 -0.0106761466194536 -0.00711743107963569 -0.121886113296851Hon. Daniel Goldman 2023-01-13 2023-02-09 27 TOSYY P ST 7500 -0.119977046121376 -0.0614236331881727 -0.0671641831899957 -0.118828914222634 -0.0360766289687555 -0.0614430759104337 -0.0918827047236124 -0.128523098218027Hon. Daniel Goldman 2023-01-13 2023-02-09 27 UL P ST 32500 -0.00537879991526324 0.0956217012718268 0.0730269472022651 0.0526239178621528 0.00874815194351353 0.0613638858330807 0.0226200470971283 -0.0286896760212338Hon. Josh Gottheimer 2023-01-13 2023-02-06 24 CRM P ST 7500 0.0822241009189104 0.170659469535108 0.278083382319235 0.690801351504753 0.144271363028414 0.297705974061224 0.520165951595339 0.800481814733304Hon. James E Hon Banks 2023-06-09 2023-06-09 0 PTON P ST 7500 0.00962694580823431 -0.299639003877472 -0.26835143000071 NaN 0.00962694580823431 -0.299639003877472 -0.26835143000071 NaNHon. Daniel Goldman 2023-06-09 2023-07-06 27 TMUS P ST 7500 -0.0180368492129533 -0.00790452082733051 0.168559862667367 NaN 0.0531364547756934 0.0450671154539486 0.188937399749666 NaNHon. Kevin Hern 2023-06-09 2023-06-12 3 EMR P ST 7500 0.0957535445819113 0.197395489056808 0.0794284372962142 NaN 0.0621139454394417 0.186949086644804 0.0534964128465782 NaNHon. Kevin Hern 2023-06-09 2023-06-12 3 XOM P ST 7500 0.000657849133166577 0.0815245327111473 -0.0475938707816278 NaN -0.0251419611488686 0.0697839895918446 -0.0610282539642848 NaNHon. Jonathan Jackson 2023-06-09 2023-07-07 28 CRM P ST 32500 0.0308698111376842 -0.0361658610571645 0.201584062017073 NaN -0.0116111468599394 0.0335330710406958 0.157075841099597 NaNHon. Earl Blumenauer 2023-02-15 2023-03-08 21 NWN P ST 7500 NaN -0.0698541895243467 -0.165367868317723 -0.163335644068299 -0.0265485987769353 -0.0833381208534622 -0.148960054738657 -0.221834310334865Hon. Virginia Foxx 2023-02-15 2023-03-01 14 ARLP P ST 7500 -0.029355224757572 -0.12228749613865 0.0385564707585095 0.0441845368864233 -0.124419703464602 -0.105200013379135 -0.00704592763200873 0.0391533508324917Hon. Virginia Foxx 2023-02-15 2023-03-01 14 FLNG P ST 7500 -0.029199015839081 -0.0994862048838192 -0.0692716536626806 -0.189419798462259 -0.0807021953346341 -0.0926062628740404 -0.0196376191133473 -0.132336260114085Hon. Kevin Hern 2023-02-15 2023-02-16 1 ACN P ST 7500 -0.101948756978119 0.0161001298138854 0.101369323546314 0.334137512009306 -0.130888138243039 -0.0255992609572688 0.0918470617600759 0.311481269350448Hon. Earl Blumenauer 2023-05-15 2023-06-05 21 NWN P ST 7500 0.00539139166912928 -0.0888043349943343 -0.0828878181485968 NaN -0.0477776764973166 -0.0905687901793475 -0.185912790751033 -0.112636050325756Hon. Josh Gottheimer 2023-05-15 2023-06-06 22 MSFT P OP 2000000 0.0227464698636466 0.00172677026673784 0.110861987789649 NaN 0.0924754044655987 0.0494034251322919 0.190007120710131 0.357134440581016Hon. Josh Gottheimer 2023-05-15 2023-06-06 22 MSFT P OP 2000000 0.0227464698636466 0.00172677026673784 0.110861987789649 NaN 0.0924754044655987 0.0494034251322919 0.190007120710131 0.357134440581016Hon. Josh Gottheimer 2023-05-15 2023-06-06 22 MSFT P OP 750000 0.0227464698636466 0.00172677026673784 0.110861987789649 NaN 0.0924754044655987 0.0494034251322919 0.190007120710131 0.357134440581016Hon. Kevin Hern 2023-05-15 2023-05-16 1 OKE P ST 7500 0.0800636456525656 0.182404766141277 0.176399916113952 0.524483687862098 0.0427955129293789 0.154451445089963 0.148588556179073 0.489719941788557For situations where there are NaN entries for one of the data items, I'd like to fill in with the entry preceding it from within its group. Groups here are columns of the form *_p_ret and *_t_ret.
So, for row for name 'Hon. James E Hon Banks', I'd like the NaN in 12m_p_ret to be populated by the preceding entry, -0.26835.
Where a NaN doesn't have a preceding in group entry (ie, 1m_* columns), I'd simply like to replace NaN by 0.
Any way to do this?