6

I have a couple of users who have a User Interface for an application that they must use to do their job. The UI can be cumbersome to navigate to perform some time critical tasks.

This UI does not expose any API and has no integration points.

I would like to be able to do something to help the users to perform various tasks.

I've been looking into the feasibility of using VS2013's coded UI tests to allow me to control the UI from some of my own code which would run on the user PCs. Obviously this is not really what coded UI tests are intended for.

Has anyone else tried this technique to integrate code with a closed UI? How successful were you? Are you aware of any other frameworks which would achieve the same result?

In some more detail (not required reading for question): The UI in question is a market making tool within finance. For simplicity sake, let's say the use of it is enforced by regulation. Currently the users will run a model and then manually type several values into the UI as their bids and offers. This is time consuming and has potential for human error (even with copy and paste). I would like to create an app that took the correct values from the model and posted them into the correct boxes on the screen.

Things I can control:

  • The application they take the values from
  • The integration software

Things I cannot control:

  • Anything to do with the market making UI

Any suggestions or advice much appreciated.

9
  • 3
    You may want to consider AutoHotKey
    – JeffO
    Apr 27, 2016 at 20:27
  • Not 100% sure about AutoHotKey but you should make sure you can use window classes to recognize controls. Don't go for timing-based scripting.
    – user44761
    Apr 28, 2016 at 6:58
  • 4
    I used AutoHotKey in the past for automating UIs. For pure scripting, however, AutoIt might be better suited. How well this works, however, depends on how stable the UI layout of the controlled application is (from one version to the next). An old list of UI robots is here: en.wikipedia.org/wiki/List_of_GUI_testing_tools
    – Doc Brown
    Jun 27, 2016 at 6:10
  • Can you decompile closed UI and check possible names for controls you need. Usually this kind of finance market software have plugins possibility
    – Fabio
    Sep 7, 2017 at 2:51
  • 1
    Is this a web UI or a desktop application? If desktop, any idea what GUI toolkit is used? Mar 6, 2018 at 14:57

3 Answers 3

1

You can probably make it work. But, I find when using coded UI, or other test frameworks you are quite dependent on changing the code to make the various buttons and other elements easy to pick out and interact with.

I think you will find it easier to just buy an off the shelf automation product

5
  • bing it! winautomation.com/macro-recorder
    – Ewan
    Apr 27, 2016 at 22:40
  • Mouse and keyboard recording is not suitable. There are six monitors on a standard traders PC. I need to be able to interacts with individual controls in a closed UI
    – GinjaNinja
    Apr 27, 2016 at 22:47
  • Essentialy this kind of software does the same thing as coded ui. sends windows interupts to a process. They have just given you a ui to record the actions. You can code it directly into a console app if you want. But its never going to be super reliable if you run it on a users pc where you dont know what windows are open etc
    – Ewan
    Apr 27, 2016 at 22:53
  • I wrote a couple of console apps to do a similar thing years ago. But you had to run them on a server with the os/desktop etc in a known state. Open two windows with the same name for example and it wouldnt know which to click. Press tab while it was in the middle of its script? Focus changed to the next button and everything goes wrong
    – Ewan
    Apr 27, 2016 at 22:56
  • 4
    @Ewan: I would rule out any tools which advertise themselves with "no programming" immediately.
    – Doc Brown
    Jun 27, 2016 at 6:05
1

Have you looked into Spock and Selenium? Spock can load pages and selenium can manipulate UI elements. This can be set up for purposes other than testing and should run just fine. Beware, things can get tricky if you have UI elements that need to be activated, then you have to wait for something to load, and then you can proceed. Some creative coding may become necessary when it comes to telling it to wait the required time.

Selenium is not generally loved by most people who have to use it a lot, but it works. It should be possible to set up a script that would run and a sequence of actions in the UI occur.

1
  • Spock is for Java and Selenium is for browser, neither seems applicable to the OP. Mar 6, 2018 at 15:04
-1

How to programmatically control a GUI application?

I guess that you have written, or at least can modify that GUI application, so have access to (and understanding of) its source code (and the capacity and resources to improve it).

Then you could add some RPC-like service to that application. You could consider adding some JSONRPC, some REST or other Web service, some SOAP, etc...

Adding such service to an existing GUI application is generally easy (since GUI frameworks have some event loop).

If you have no control (and no access to its source code) on the GUI application, in general you cannot control that GUI application programmatically. But depending on the display server (X11, Wayland, etc...), the operating system and the widget toolkit used by it, in some specific -and rare- cases you might control it programmatically (for example, some recent GTK applications using Gobject introspection).

Perhaps the best solution is to go back to the vendor of that GUI application and ask him (and pay him) to develop some programmatical interface (and discuss with him the technical details, e.g. the inter-process communication protocol to use).

If you know the source of the data ("The application they take the values from") and the computation that this GUI is making, you might perhaps consider redeveloping your code doing that computation (Only you could know if that makes sense).

Read also Comparison of GUI testing tools and GUI testing wikipages.

2
  • As mentioned in original post, I have no control over the GUI whatsoever. I don't specifically mention it is provided by a thord party.
    – GinjaNinja
    Apr 10, 2018 at 8:46
  • 1
    @GinjaNinja: why don't you answer my question? Do you not want to get a useful answer? Apr 10, 2018 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.