Seeking advice on design strategy for Java application

Question

I'm a very new programmer developing my first application in Java as a side project for my employer. I'm a part-time student working full-time hoping to eventually score a developer job, and my employer encouraged this by allowing me to set aside some time daily to work on it. For the sake of anonymity, we'll call my employer ABC.

Problem Background

ABC stores documents electronically, and receives many electronic documents each month. ABC also receives a fair amount of paper documents each month. Some of these paper documents are duplicates of electronic documents received and shredded once they've discovered this. However, some are found to be novel (not duplicates!) and need to be manually entered (read: hand-keyed by data entry workers) into the e-document system, scanned, and then shredded.

These documents come from different sources and can have slightly different formats, but contain the same information that is going to be input. Data in received documents can be used to distinguish between unique and duplicate documents - specifically, dates and other values. If enough of these match to an existing document (easily implemented in an matching algorithm that scans a "document group overview" page), then the current document can be marked as a duplicate and skipped.

The e-document system is a locally-hosted server, but accessed through a browser. Documents are stored in this system are grouped and organized based on specific associations, tethered electronically via an ID number that is used internally - but not present on the documents when received from external sources.

Use Case

I decided I wanted to develop an application that would automate this process with four phases, with the following tools in mind:

1. OCR of the paper documents after an initial scanning (Tesseract/Tess4J)
2. Identifying necessary data during OCR and storing it locally
3. Identifying if the paper document is novel or duplicate (Selenium)
4. Automatically entering novel documents into the system (Selenium)

Classes

My intent is for no less than three or four classes:

• A: an OCR/analysis class that fulfills phases 1 & 2
• B: an automation class that minimally, fulfills phase 4, possibly phase 3 also
• C: an initialization/control class that performs housekeeping tasks
• D: (optional) a search/identification class that fulfills phase 3

A calls a Tesseract instance on an image file passed in from C, analyzes the document line-by-line, and stores data in local variables to be passed into the automation class.

This class is fairly straightforward - the information that I want is the same from document-to-document. The only complicating factor involved is that documents received from different sources come in different formats - the data of interest is stored in different places. Making A an abstract class (with Class extensions for each document source) is a possible solution here, but it would have to be fairly simple. Tesseract processes images line-by-line, and since each line will have different information depending on the document source, processing methods will need to vary for each.

Is this a good approach to this class?

B is a potentially complex class, and one I've been struggling with in regards to design strategy. It needs to perform the following functions:

• receive passed-in variables from the OCR/analysis object (analyzed image file)
• **automate searches for the document group the analyzed document/images belong to
• **identify the instance of a duplicate document, and in the event of such, stops the automation process
• perform basic navigation functions (to search pages, to "add document" pages, etc)
• navigate to each sub-page of the "add document" page
• enter information belonging on each sub-page via forms/selectors

I'm struggling with this in regards to methods/class design. There are a lot of repeat behaviors (ie: lots of selectors, fields accessible/unblurred only after a selector value is chosen) as well as a select few of random behaviors (a field that generates a JavaScript popup after a value is keyed in, in which a link must be clicked). There is also a key data type/sub-page that, based on information within the paper document, will vary in terms of fields available.

Is it possible to produce generic methods (ie: a selector method for selector/blurred field combinations) that can be used broadly? For the variable sub-page, would it be considered bad practice to put all of that automation code into a single method? Also, should I separate out the search/identification functions into D?

C is our initialization/control class. This does all of the "fun" housekeeping tasks.

This class sets the Selenium WebDriver parameters, starts a webdriver instance, and closes the instance when the tasks have been completed. It will perform basic navigation actions such as the initial navigation to the user login portal, where it will wait for the user to login. It should likely contain (or initialize from another class) a Swing GUI file chooser, enabling our non-computer friendly end users to easily select the document image files to be passed into A. It should contain a control loop that enables the user to scan multiple documents in a single login session (maybe with a GUI using a selector or form to tell the application how many documents are being scanned in the current session).

This class should instantiate objects from the OCR/analysis class and the automation class for each image file selected. I'm not really sure of operation time requirements yet, so there may or may not need to be a wait to allow for processing between analysis and automation. There likely should be a wait for the automation process to complete before analysis of a new image begins.

This class has the potential to be quite large (but nowhere near as large as B), so I'm not sure if I should break it up. I definitely could use some suggestions on that.

Again, feel free to offer suggestions for anything and everything. I apologize in advance for being vague and not presenting my current code - the data stored in the documents contains confidential information, so I'm trying to avoid identifiers. The code for the web application that accesses the electronic document system is also not public, so by revealing my code I might be breaking confidentiality rules that have not been fully explained to me yet. If I get the green light from my supervisor on any of this, I will share what I'm able to.

Answer 1

The road to Damascus was built in sections, and so shall yours. No one builds a 500 mile road all at once, all or nothing.

Beware the Dunning-Kruger Effect

---

Define data formats. Sometimes they're not obvious. These will always simplify the using code. Even small and simple structures. Data will be a primary way the software parts will communicate. And I don't mean just those classical structures we learn in school: tree, stack, queue, etc.

---

• What is an abstract class?
  • Is it possible to produce generic methods ... that can be used broadly?
  • There are a lot of repeat behaviors

---

My intent is for no less than three or four classes Those naive of OO design are easy to spot. Upon seeing a design they immediately say "that's too many classes."

---

B is a potentially complex class

Good OO design/programming manages complexity by making simpler things that interact. "B" is not one class.

The Single Responsibility Principle is very, very powerful but can be groked only through diligent practice; constantly looking for coalescing abstraction and subtle concept.

---

a field that generates a JavaScript popup after a value is keyed in, in which a link must be clicked

This is an implementation detail. At design this is virtually irrelevant. At this point in your design any user interactive GUI is virtually irrelevant. Sure, think about it. Even sketch some of it out. But do not let it drive design.

---

Wishful thinking design

Design what you need. Then design whatever it takes to interact. For example:

• Here is a class: the information that I want is the same from document-to-document.
• Here are several classes: documents received from different sources come in different formats
• Now integrate. Possibilities:
  • Inheritance
  • Class(es) that transposes data

---

Spend LOTS of time in design

... and then spend some more.

True story. Creating 2 from-scratch pieces in the same larger project. "A" was very thoughfully designed. Everyone, especially management, said we're taking too long designing. And lots of "there are too many classes." "B" was designed fast, fast, fast because "we had no time."

Final results:

"A" went through formal testing in 2 days. "B" was thrown into testing before coding was complete because it was proving very hard to code; it was a self reenforcing "we had no time." Then it spent 3 months in a "test-fail, fix/hack, retest-fail" cycle of hell. No kidding, 3 months.

Analysis

Coding "A" was fun. Coding "B" was torture.

Make no mistake about this. "A" was objectively much more comprehensive and complex functionality than "B". NOT THE CODE, the functionality.

"A"'s design expressed the business domain in business domain terms and concepts. We played "what if" and "how will this work" a lot before a single line was written. Function was integrated/fitted to the desired GUI. "B" was designed only based on the various sections of the visual form. There was no regard for the underlying business domain.

Integrating the bits of "A" was easy because they were all commonly understood business classes. Integrating "B" was like a whole extra effort of hacky patch code because everything was arbitrary grids of numbers and individual, seemingly arbitrary, values on the screen. There was no coherence behind what the screen displayed.

"A" was heavily unit tested. "B" had none. Indeed, coding "B" was so hard that "we had no time for writing tests."