How I Built My First Desktop App in 3 Days

“Twelve scanners! For that Sheraton job there were three scanners and it was too much work to review properly. The workload is going to be insane.”

Homeboy, the pessimist he is, was highlighting the downside of his company’s most recent contract. To be fair, he was one of the supervisors on the Sheraton gig and was literally drowning in the work, so the pessimism wasn’t unmerited.

I’d dropped by to return a flash drive and somehow ended up in a lengthy discussion about his challenges at work.

“What makes a supervisor’s role so hard?”

“We need to make sure the scanners are doing their job correctly. We were excited about almost completing the last job when we discovered a problem two scanners made weeks ago. We had to spend an extra month fixing it up.”

“What exactly do supervisors do?”

“We check to see if the scanned documents tally up with the report sheet the scanner produces. A document is a collection of PDF files. We make sure the PDFs are in the right document and have the right number of pages.”

“That sounds tedious.”

“In a week, a scanner can generate up to a thousand PDFs. It’s very easy to make a mistake and put a file in the wrong document or write the wrong page tally. It’s our job to catch and correct those mistakes.”

I stood in silence as I absorbed the gravity of the situation. 1000 PDFs per scanner is a frightening amount of data to manually verify. Pessimism must be contagious because I felt myself coming down with it. Making software is tricky, but it certainly wasn’t a tireless grind like this.

Maybe I could build something that could help him. He could select the folder containing the documents and it will determine what PDFs are in what documents, how many pages each PDF has and generate a report.

“I’ll build something to help you this weekend. Don’t sweat it”.

“Dude I’ll never ask you for your sister’s phone number again in my life if you do”.

PLANNING THE APP

I’ve only built web and browser apps, but everything about this sounds like a desktop app.

It can’t be a browser app. There aren’t APIs in the browser that allow me to crawl a client’s system. It can’t be a web app either. Uploading that many PDFs is seriously time-consuming and data intensive.

If I could get Node.js on his system, I can treat his computer like a server and build a web interface for him to interact with. Made for his computer, using web technology.

My search led me to Electron. Desktop applications built with Node.js? Everything I need to build something for a desktop with tools I use daily.

Getting “Hello World” up and running in Electron was a piece of cake. The documentation gave me enough information to do so on the first page. The code was easy to reason about and immediately dealt with the finer points of what differentiated electron apps from regular Node.js apps.

There’s a main process and a render process. Think of main as the server and render as a browser.  In a traditional web application, all the brains are on the server and the client browser talks to the server by sending text via HTTP. In Electron, the render process talks to the main process by sending text through a package called ipc.  You coordinate both processes via ipc to accomplish your object just like the browser and server are coordinated via http requests.

Once I understood this much, the plan was crystallized. Use node’s fs package to do the crawling, use comma-separated-values to generate the report excel doc and use the interface to collect input from Homeboy.

Time to start wiring this thing up!

BUILDING IT OUT

Being able to drag and drop the folder containing the documents would be most intuitive way to get the document location, so I started building from there.

Normal browsers have a security model that prevent you from getting a file’s full path with the File API, but it was there for the taking in Electron.

Used ipc to send the paths to the main process and started writing the script to crawl the paths.

The first version I wrote was procedural. Get a list of folders and retrieve their content.  If any are PDFs, report them. If they are folders, go into those and do the same thing.

No matter how deeply nested the folder structure was or how files there were, it returned the information in less than a second. Amazing fast!

Every virtue the first version had was made irrelevant by how unusable the user interface was. It had one line of unstyled text that said little more than “drop folders here”. When you do, it creates some barely-styled boxes with the folder path, number of files and number of PDFs. It worked though, so I was super excited to demo it to him.

FIRST DEMO

Homeboy watched it run through the test data and display the results.

“Can it tell me how many pages are in the PDFs?”

“That’s the very next feature I’m adding.”

“Okay. It needs to be able to tell me how many pages are in the PDFs and create an Excel sheet with that information.”

Was it too much to expect a little gratitude at this stage? No matter. I’ll just use a package from npm to get the page count. Shouldn’t take me more than 15 minutes max.

Two npm packages and as many hours later, still no page counts.

PDF PAGE COUNTS

My first attempt was using an aptly named package: pdf_page_count. The package kept returning with “undefined” no matter what I did. Homeboy’s periodical “is it done yet?” didn’t help matters. After taking a look at how many people downloaded it recently, I came to the conclusion that the package was broken. Needed to find an alternative.

pdf2json seemed to fit the bill. Way more uses and downloads than pdf_page_count, so I installed it and gave it a shot.

No such luck.

I was so close! All I need is the page counts and I can output out on the interface and generate the excel report.

SO. CLOSE.

It didn’t help that the instructions for pdf2json  were woeful, so it felt like I was listening to the wrong event or setting the listener on the wrong object or something.

In my desperation I started stepping through the source code and adding log statements so I can follow the process through and see what’s wrong.

I was listening to the wrong object. The right one nested inside. Quick change, restart app and … nothing.

Turns out the information isn’t passed to the nested object. You have to check the outer one again. Quick change, restart app and … still nothing?!?

SO. FUCKING. CLOSE. WHY ME? WHY IS LIFE SO HARD? WHY DO BIRDS SUDDENLY DISAPPEAR? WHY IS JADAKISS AS HARD AS …wait a minute …

Oh wow. Turns out I was only passing the name of the PDF, not where it was located. Quick change, restart app and …. PAGE NUMBERS!

Quickly exported the app to the other laptop for him to test and proceeded to watch it hang up his computer for five minutes before producing the report.

“Is it exporting to Excel yet?”

BACK TO THE LAB

Though I was able to extract page numbers, I was completely unsatisfied with its performance. It took an unreasonably long time to get.

To make matters worse, it completely hung up the system while getting the page counts. Forget multi-tasking. The mouse barely responded to movement and you certainly couldn’t click on anything.

pdf2json grabs a whole lot more than page counts. It tells you the height of each page, what is contained on every line (vertical AND horizontal), text and so much more. Useful information for someone else but totally irrelevant to Homeboy’s needs.

I took another look at pdf_page_count, armed with the knowledge that its failure was most likely due to me not passing the right page path.

A quick swap later it was running correctly. Way faster too! Understandable, since it’s a one-trick pony. Or a one-trick Veyron, speed improvement considered.

Processing the PDFs still hung up the system though. The app would navigate to the folders and go as deeply as it could, sorting between directories and files and trying to get the page numbers from the PDFs. My sample data has 2999 PDFs in total and the current architecture processes them all at the same. No wonder the system becomes completely unusable.

I created a queue where the PDFs to process are placed. I instructed the app to process a maximum of 8 at a time (get it? 8 looks like an upright mobius strip). The end result is not as many PDFs are processed at once. Definitely wouldn’t be as fast as opening all 2999 at once, but the system doesn’t experience any performance degradation. He’d be able to drop a bunch of folders to be processed and can muck around on Facebook, check back in 5 minutes and it’s all done.

UGLY DUCKLING TO SWAN

I could have aimed for feature complete at this point and done the Excel exporting but the unusable user interface bothered me far too much. I knew when the app was processing and when it stopped simply because I had a console attached to the app. This shouldn’t be background information. This is data the app user needs to know, even if they don’t know they need it.

I wired up ipc to log how many PDFs were queued up twice in a second so the user knows what’s left to process and how fast the system is running through it. I put an indicator at the top left that’s green when there is no processing going on, red with a pulsing orange light when its busy. A quick glance should be good enough to let you know what exactly is going on.

Not quite the most beautiful swan, but this was a Sunday afternoon. Being feature complete at the end of the day was far more important than making it the most pretty thing. Enhancements could wait.

FEATURE COMPLETE

I left the Excel report til very late because I’d just worked on a Meteor app with Adim and a feature I worked on was exporting datasets to Excel. My understanding of the problem space and what it takes is still very fresh.

Generating the data wasn’t a problem, but the method of export is slightly different than it would be in a browser. Data URIs were not working in Electron, so I pushed the data back to the main process and wrote the file out with Node fs package.

Admittedly the workaround didn’t come to me immediately but I realized I’m not alone, searched and found a post online detailing the problem as well as a possible solution and figured out exactly how to do it in my app. Problem solved, feature complete.

FINAL DEMO

“Duuuuuuuude. This thing is awesome! You don’t know how much easier you just made my life. Thanks a lot man!”

Might be because I was expecting some pessimism but I was extremely pleased to hear how happy he was with it. He’s taken it to his office and is currently giving it out to his coworkers to use. I’m totally floored.

There’s only a handful of people using things I’ve made or libraries I’ve written, so every additional person enjoying what I’ve worked on brings a feeling of joy and motivates me to work harder to build more things.

Definitely going to be build more things and writing about it 🙂