-
-
Notifications
You must be signed in to change notification settings - Fork 39
Developer Diary
Decided to download some NYC taxi data and ran it through converting to disk.frame. It ran remarkably well with no issues. But then I realized the map actually had parallelism disabled as it was using lapply only, so I switched it back to future_lapply.
I then decided to double-check the Fannie Mae tutorials are still working. And then I realized that I can save so much code now that my csv_to_disk.frame can naturally handle multiple CSVs in one go instead of having to manually do a rbindlist.disk.frame. As I was running through the code I realized how a lack of progress reporting was making the user-experience less than ideal, so I tried to use .progress implemented by furrr to display basic progress bars, like this
Converting CSV to disk.frame: Progress: ────────────────────────────────────── 100%
Fixed more issues with the CRAN submission including converting all cases of T/F to TRUE/FALSE and making sure that I use message() which I had never known about instead of print() for printing user messages. There were some other fixes including not writing to user's filespace and write to tempdir() only. Also, I cannot use more than 2 cores, so I had to comment setup_disk.frame() as that may use more than 2 cores.
After fixing all that I went to submit but was greeted with
Submission server vacation from on Aug. 9, 2019 to Aug 18, 2019
During this time, the submission of packages is not possible.
Sorry for any inconvenience
This is OK. I will just submit one week later.
Got a maintainer email from Martina Schmirl. I need to reduce the title to less than 65 characters and there were a number of requirements for running the examples.
Fixing them now.
Adding examples to every function. BORING!!! But I think will be super-useful for other users.
Submitted to CRAN for the fourth time. I feel sorry for the amount of volunteer time I have already taken up. The "sin" I committed was writing to user's directory and not writing to tempdir(). I have fixed this. Along the way I have added the default outdir as writing to somewhere in tempdir() so the user doesn't have to specify outdir = every time.
I am also tempted to make overwrite = getOption("disk.frame.overwrite.default") and set the option to TRUE by default. As I find setting overwrite= annoying. But then again, people use disk.frame to manipulate large amounts of data so it's better to be safe and default to overwrite = FALSE.
I have submitted to CRAN but then realised that setup_disk.frame(gui=T) does NOT work if you started a fresh session of R