6

I am rewriting a streamlit app that is an interface to a laboratory management system (LMS). This means, that I have to make a lot of requests to that LMS through its python library (PyBIS). The way that library works is, that I import a class and instantiate it, and then call a method to login to the server. This login process however takes a couple of seconds. This means, I want to call the "login" method only once, and use that instance, lets call it pybis_instance, for all of my following calls.

Now to my issue: I have about 10 different functions which I use to get data from the LMS. They all need that instance I mentioned before, to get their data. Almost all of my other code uses one of those 10 functions at some point. Now, I have two options that I know of to pass that instance to those functions:

  1. Pass that instance as an argument to the function. This was my first way of doing this and seems like the "right" way. However, now my code is cluttered with the pybis_instance parameter, almost every function needs it, mostly just to pass it on to the next function which is called inside.
  2. Use a streamlits session_state to store that instance. This makes things a lot easier, since now I only need to care about pybis_instance in my 10 functions which interact with the LMS directly. However, the session_state function of streamlit is kind of the same thing as a global variable, with the exception that it is namespaced into the streamlit package. And I've always learned "global variables are bad, don't use them".

Now, is there a good way to 1. not clutter my code with the same parameter/argument in every function and 2. not use a global variable instead? I can't come up with a good solution to this. I thought about wrapping my instance and my 10 functions into their own class and to just be able to import that, but for the login I still need to instantiate that class and still need to pass that exact instance around. Maybe there's some way to write a helper function which checks if an instance of that class already exists?

At the end some code examples to illustrate how 1. and 2. work at the moment:

1 - instance as an argument

from pybis import Openbis

def login():
  pybis_instance = Openbis(URL)
  pybis_instance.login(USER, PASSWORD)
  return pybis_instance

## in app.py:
pybis_instance = login()

create_some_interface(pybis_instance, ...)

## in interface.py
def create_some_interface(pybis_instance: Openbis, ...):
  ...
  some_other_routine(pybis_instance, ...)
  ...

...

## in get.py
def get_some_data(pybis_instance):
  return pybis_instance.load_data()

2 - instance in global variable

## in app.py
st.session_state["pybis_instance"] = login()

## in get.py
def get_some_data():
  return st.session_state["pybis_instance"].load_data()

Edit 1

would it work to assign that instance to a variable and then to import that variable into other functions? is there some caveat?

## app.py
instance = login()

## get.py
from app import instance

def get_some_data():
  instance.load_data()

-> Global Object Pattern

However, streamlit re-runs app.py all the time - which would cause the code to execute. If I assign a variable in a module, is it executed only once? Maybe I could put instance = login() at the top of get.py? -> sadly doesn't work, I need to provide login details, so I can't instantiate at import

Edit 2

Also, maybe a singleton pattern could work here? Singleton

4
  • Step away from the cliff. Globals are not the way. "almost every function needs it, mostly just to pass it on to the next function which is called inside." Why not pass your object as a parameter to a constructor of a new class, whose 10 methods can access it as an instance variable, with no need for explicit params?
    – Alexander
    Commented Oct 13, 2023 at 14:42
  • Yeah, that's what I thought about globals. Your solutionis something I thought about (and tried) as well. My problem is, that I have a bunch of other functions who would need to call those 10 functions/ then 10 methods. So for that to work, I need to pass in the instance of that new class to those functions. And at that point I am back to the same question - do I pass it as parameter or somehow else? My issue is also, I need to somehow get the username and password inputs from my interface to that class - so I need to instantiate it in my main app, which streamlit heavily reruns all the time.
    – Jan
    Commented Oct 13, 2023 at 15:10
  • Is there a reason all of your functions are top-level declarations? Put them as methods in a class and there will be a natural place where to put the pybis_instance
    – Bergi
    Commented Oct 13, 2023 at 20:51
  • @Bergi Well in python namespacing with modules works quite well, so I don't use classes just for that purpose. But you are right, in this case classes would probably help to provide an interface for pybis_instance
    – Jan
    Commented Oct 14, 2023 at 9:20

2 Answers 2

5

The main concern I have with all the proposed solutions is that I don't see where you are considering the potential for the pybis connection to be invalidated. I am not familiar with this library but a cursory glance at the PyPI implies it is on you to manage. It might be worth searching for a connection pool that could help with this.

First of all, I think we need to be clear about what is meant by 'global' in Python. The word 'global' has a lot of negative connotations for programming in general but the meaning in Python isn't exactly the high-level abstract concept that the name implies. In Python, 'global' essentially means module-level scope.

Based on what you have described, I would lean towards keeping it simple. Create a module which is dedicated to managing the pybis dependencies. Then you can create a module like so:

from pybis import Openbis

def _login():
  instance = Openbis(URL)
  instance.login(USER, PASSWORD)
  return instance

_pybis = None

def session() -> Openbis:
  global _pybis
  
  if _pybis is None or not _pybis.is_session_active():
    _pybis = _login()
  
  return _pybis 

Note: I have no idea what kind of transaction boundaries there might be in pybis and how those relate to sessions. That kind of challenge is outside the scope of this answer, but this approach can give you a starting point for dealing with that kind of complexity.

Then in your other modules you can import your module and call the session() instance whenever you need pass an instance to something else.

I'm going to guess that some people might look at this and think this is basically a Singleton. That's true to a point but I don't see any major reason to make it more complicated. If you only ever need a single live instance of OpenBis it doesn't make a lot of sense to write a bunch of code to avoid that for no reason.

The next question is likely "what about testing?" In Python, this isn't much of a challenge. In your testing setup, you replace the session method with a fake or a mock. e.g.:

import sys

sys.modules['pybis_manager'] = fake_pybis_manager 

There is also nothing preventing you from using DI principles to inject the method reference to session where it is needed.

It's possible that this solution is inadequate for reasons you haven't described so let me know if there are challenges that you see with this.

5
  • Thank you for your extensive reply! I like this solution, I was trying to adopt something similar (see Edit 1, by far not as clean). My issue was, that I'm having difficulties seeing a good way to provide username and password to the function. This is a bit outside of the scope of my original question, but in my app.py I am asking the user for their credentials through a streamlit form. So if I imported session to app.py, I would need to add those credentials as a parameter to session() - but I only need that info when I'm authorizing. So mayb a class based approach could work better?
    – Jan
    Commented Oct 13, 2023 at 16:56
  • Of course I could also take another route and save the user credentials as module variable in app.py as well and import them to the session module. But saving credentials to a variable somehow feels bad for security reasons? Silly I know, since everything is just in memory anyway, and if something goes wrong the credentials can show up in the stack trace either way.
    – Jan
    Commented Oct 13, 2023 at 16:59
  • 1
    That is a whole can of worms. As you note, to capture the credentials in variables a reuse them. There are potential security concerns with that. You could also prompt for credentials whenever you need to connect or reconnect. As noted in the docs: "In an interactive session e.g. inside a Jupyter notebook, you can use getpass to enter your password safely:" IMO, though, if that kind of thing is really a concern, simple user/pass security should be replaced with something more robust. The only other option I see is to use a PAT but that doesn't seem to address your concern.
    – JimmyJames
    Commented Oct 13, 2023 at 17:08
  • Yeah I know, it somehow is more complex than it should be. Part of the problem is that I'm using streamlit - it turns out that a simple top to bottom execution is not always suited for web applications. But I'll try to implement something that reprompts the user. Thank you for your efforts!
    – Jan
    Commented Oct 13, 2023 at 17:15
  • 1
    This might help: docs.streamlit.io/knowledge-base/deploy/…
    – JimmyJames
    Commented Oct 13, 2023 at 17:58
3

This is where session management comes into play. Think of this "login" step as initiating a session. This session is common to all other code that is required to interface with the LMS library. Encapsulate this behavior in one or more classes and pass a session as a constructor parameter. The session can lazily log in.

class LmsSession:
  def __init__(self, url, username, password):
    self.url = url
    self.username = username
    self.password = password
    self.pybis = None

  def __pybis(self):
    if not self.pybis:
      self.pybis = OpenBis(self.url)
      self.pybis.login(self.username, self.password)
    return self.pybis

  def do_something(self, arg1, arg2):
    self.__pybis().do_something(arg1, arg2)

class SomeSpecificKindOfStorage:
  def __init__(self, session)
    self.session = session


  def do_something(self, arg1, arg2)
    self.session.do_something(arg1, arg2)

Let the session initialize itself so consumers of that session do not need to. Encapsulate this behavior, and create additional dedicated classes that provide a domain-specific interface for doing meaningful work in your application. Basically, replace SomeSpecificKindOfStorage with a name that means something for your application, and put methods in it that work well together. This will hide the LMS session entirely from the rest of your application at the expense of initializing LmsSession once, and passing it around from constructor to constructor.

# app.py
session = LmsSession('https://...', 'username', 'password')
storage = SomeSpecificKindOfStorage(session)

storage.do_something(1, 'foo')

This avoids the need for repeated method parameters and global variables. See also Dependency Injection.


I'm a little light on Python knowledge, and I'm sure there is a cleaner, more pythonic way to write the __pybis method.

6
  • Thank you for this in-depth explanation! In LmsSession, in __pybis, I would need to add a return self.pybis, right?
    – Jan
    Commented Oct 13, 2023 at 15:02
  • and what I get is, you would propose to wrap all other parts of the code which need to access session in fitting classes, right?
    – Jan
    Commented Oct 13, 2023 at 15:05
  • 2
    Or to put it differently: embrace the dependencies, but use classes to not make it a mess? :D
    – Jan
    Commented Oct 13, 2023 at 15:12
  • 1
    w/ regard to __pybis: the keyword not is used instead of ! and it's not an existential operator so you need to do something like self.pybis = None in __init__
    – JimmyJames
    Commented Oct 13, 2023 at 15:14
  • 1
    @Jan: if you log in while executing the constructor, it will take several seconds just to initialize an object. Object initialization should happen quickly if at all possible. What if you initialize the object, but you don't end up calling out to LMS? You incur a performance hit for no reason. That's why it's usually best to make web service calls only when necessary. Commented Oct 13, 2023 at 18:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.