And I would argue that the code used may not be the most optimized, but at least gives a gist of maybe what a typical approach and implementation to the problem of splitting strings on a large file might entail. Below are some of the other pertinent control variables: Operating Systems Windows 10, Kali Linux WSL Memory 32 GB CPU Intel i7-7700k HDD m.2 1TB SSD Notable Background Programs Chrome, Slack, Outlook, Notepad++, Spotify (gotta jam while we wait!), etc.įinally, I would attempt a final test that would parse and cut the large 27GB file (500M lines), since that is really why I started this in the first place. Also, I wanted to ensure read/write times of my separate hard drives wouldn’t accidentally impact the test, so this smaller file ensured I could write numerous files to my SSD. I also decided after a few initial attempts, that the initial test would be on a smaller subset (but still rather large) of the first million leaked passwords, which resulted in a text file of about 110MB. I came up with following: Time to complete task, time / Measure-Command CPU/Memory impact, Resource monitor / htop I then, needed to come up with measurables and ways to, well, measure them. NET sed / sed -u terrible attempt at VS c++ Golang Search / Replace in Notepad++ (RIP) cat | cut awk My short list of things to try then came to: python powershell /.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |