0

Problem:

There are proprietary algorithms that developers who are actively working on a project will need to call but they should not see the underlying logic. They need to be stored in an area of Source Control that a limited number of developers have access to. The base application will handle user input, data access, and communication with proprietary algorithms when necessary. The proprietary algorithms need to have their logic hidden so if they are included as a reference library then they will need to be obfuscated.

Environment:

  • Visual Studio 2022
  • BitBucket
  • Jenkins
  • Windows Server/IIS

Requirements:

  • Proprietary Algorithms are stored and maintained in a separate secure Source Control repository
  • Any references to the algorithms should hide the underlying logic. A DLL reference would need to be obfuscated first
  • Proprietary Algorithms may need to be modified and included as a part of software releases

Solutions:

I'm open to other solutions, but below is what I've been able to come up with so far. Hoping others can provide recommendations between the two approaches or provide additional solutions.

  1. Compiled Library Reference
    • Create a separate secure source control project.
    • This project will have its own branching structure.
    • It is only included in the project as a compiled reference that has gone through an obfuscation process.
    • I'm personally not familiar but believe there are ways to include this in the build process so it will compile the other repository and apply any obfuscation. This can get complicated though as depending on development, we may need to include a specific branch that has the features being worked on.
    • I've seen that it's still possible to reverse engineer obfuscated code so it would not be 100% secure.
  2. Seperate APIs
    • Create a separate secure source control repository
    • This project will have its own branching structure
    • API's will be deployed as their own site.
    • The base/main application will access the API's by making REST calls providing the necessary input that the API's need to perform any logic. The API's will not handle data access.
    • I like the clear separation in this approach and that it removes the need for obfuscation and makes the build process less complicated.
    • It will require the API's to be hosted separately and developers to know which API site they should be calling for the project they're working on. However, this logic will not be changing as frequently as the rest of the application so it's likely that we could have many releases that use the same API version.
    • The APIs and application can be hosted on the same server so though it would add additional overhead to make the requests, I don't anticipate it being much of a performance impact.
    • Requires more steps in the deployment process whenever the API's also need to be updated

Review

I lean towards the Separate API's architecture as it removes the need for obfuscation, increases the security of proprietary algorithms from reverse engineering (assuming the developers do not have access to the server), reduces complicated build processes that are harder to manage and removes the need for the main developers to understand and update the build process to the correct API's branch for any release. With the proprietary API's being separated, this part of the process is more clear as to who is controlling and deploying it and that team member will know the branch which should be included for any release.

Curious if anyone else has encountered these requirements for code structure and how they accomplished it, as well as thoughts on the above suggestions.

5
  • "ensures 100% security of proprietary algorithms from reverse engineering" No, it does not, particularly if you then host them on the same server and you don't trust your developers. Feb 21 at 15:43
  • @PhilipKendall Fair point....I will adjust that. However, if the developers do not have access to the server, how could they reverse engineer an API end point?
    – Rudazzle
    Feb 21 at 15:51
  • Do your developers have permission to deploy arbitrary code to the server? If so, they have access. Feb 21 at 15:58
  • Deployment is done via Jenkins, not manually by developers. The developers that the code is being hidden from don't deploy via Jenkins either. Do you have experience with similar setups or suggestions for an approach to accomplish the design? Thanks for responding!
    – Rudazzle
    Feb 21 at 16:03
  • 3
    @Rudazzle: It appears you are defining "reverse engineer" as "knowing exactly how it works internally". I just want to point out that as long as you can replicate the public behavior, you've "copied" the logic without truly understanding it. Reverse engineering implies a shortcut by finding the logic so you don't have to brute force testing the public behavior (recording every input/output), but the latter is just as much of a risk for your proprietary logic, and not even an API will protect against that.
    – Flater
    Feb 22 at 0:29

2 Answers 2

5

Simply treat your "proprietary algorithms package" as if it was developed as a closed source software by a 3rd party vendor, decoupled from your main project. You may have already such other 3rd party libs or APIs included in your project, so if you know to handle those, you also know how to handle this case.

There are things you would expect for such a package by an external vendor. Make sure you provide these things in a similar manner, like

  • proper versioning and documentation (ideally semantic versioning)

  • backwards compatibility over certain periods

  • a change log which mentions "breaking changes".

Whether you go for the "separate (web) API" approach, or the "obfuscated library" approach - or maybe both - is something we cannot evaluate for you. This is a decision only you can make by making a proper cost/benefit analysis and - not to forget - a risk analysis. A separate web API is - in theory - more secure, for exactly the reasons you mentioned, but whether you need this extra security, and if that means extra effort in your environment - or not - is something only you know.

It seems you already know the pros and cons of both approaches. What we cannot tell you how much damage you expect from devs who reverse engineer your specific package, and how high you see the risk that this will happen really. But since you are talking about devs working for your organization, you may also consider to implement some extra contractual measures, asking every dev who has to work with the proprietary package to agree to a special NDA or something like that. If this is secure enough for your situation, however, is something you better discuss with your boss and/or a lawyer.

4
  • Thank you! The 3rd party vendor comparison was a very good way to think about it. Are there any other implementation options that I'm missing or are the 2 listed the only ones that you are aware of?
    – Rudazzle
    Feb 21 at 18:05
  • @Rudazzle: I am quite sure those two are the main options you have. Either you provide your "secret algorithm package" as a binary, or not. If not, you need to provide a network API to something you host by yourself - that's it.
    – Doc Brown
    Feb 21 at 22:15
  • I could make a substantive argument that the definition of a "party" (as in "third party") inherently entails the kind of boundary that OP is putting up between the proprietary logic and its consumers, regardless of whether both of them are on the same company's payroll or not. The assumption here is that a "party" has full freedom of information within itself.
    – Flater
    Feb 22 at 0:33
  • @Flater: sure. Simply put, my answer is a recommendation to embrace the OP's knowledge about external closed-source 3rd party packages and apply this knowledge to their case.
    – Doc Brown
    Feb 22 at 7:30
2

With regard to Flater's comment, if the algorithms are so secret that you don't regard your ordinary staff NDAs and security practices as sufficient (an extremely high bar!), then one advantage of the "secure API" approach is that you can (and probably should) audit every single call made and look for unusual access patterns. Otherwise you have to consider the risk of someone "blackbox" reverse engineering it through supplying combinations of input data.

You may also want to consider a "dev/test API" which is available to the developers, but either does not use live data or does not use the exact same algorithm as the live deployment. That does run a higher risk of failures in live deployment, though.

(It's quite common for payment APIs to provide a "fake bank" API that just approves all payments, for example)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.