0

I'd like to deploy my code to a web server as a package. There Python is subprocessed and scripts are executed. These scripts use data files. But in case a file is needed and it doesn't exist, it should be created.

I intend to use pkg_resources for managing those files inside the package, but it comes with limitations. There is no way of creating those resources.

One could, of course start manipulating path variables inside the package to get absolute references to files, but I'm looking for something a bit more convenient. Not the best of reasons to not use some method, but someone might actually know something I don't!

So, what is the best or current way of creating data files inside a package?

If it makes a difference, the files are directly under the sources that manipulate them. I'm new to packaging and releasing Python code, so any help is greatly appreciated! I'd also be open to changing the approach if that's what is needed to make this work.

Improve this question
5
  • Can you clarify why you'd like to dynamically create pkg_resources? That mechanism is intended for (read-only) datafiles that should be installed together with your package, not as a work directory for your program. E.g. I use pkg_resources to bundle template files with my package.
    – amon
    Commented Jul 19, 2018 at 9:50
  • @amon Yes, I decided to deploy the code as a package to ease installation to the server (as to not worry about manually moving stuff and setting up paths etc.). The code itself has to manipulate data, and the files are mainly for serialisation. I understand the intended use, but I'm pretty inexperienced, so if there is a better way of achieving this than packaging, I'd love to hear it!
    – Felix
    Commented Jul 19, 2018 at 9:55
  • These files that will be manipulated – will other processes consume or provide those files? Do the files have to be persistent, or would a tempdir be sufficient?
    – amon
    Commented Jul 19, 2018 at 9:58
  • 1
    @amon They are for long-time serialisation, but the Python process itself will create and read them and only Python will do that. That's the reason I want to hide away the file structure from the calling server.
    – Felix
    Commented Jul 19, 2018 at 10:00
  • Could I improve the question in some way, in response to the down vote?
    – Felix
    Commented Jul 19, 2018 at 10:48

1 Answer 1

Reset to default
0

It seems pkg_resources can be used for the purpose after all. In addition to files, one can also specify a folder to be fetched as a resource. This allows pointing to that folder for any file operations.

The code could look something along these lines:

import pkg_resources
folder = pkg_resources.resource_filename('module_name', 'folder_name/')
print(folder)

> 'C:\Absolute\Path\To\module_name\folder_name\'

This works at least for the local, symlink package I've set up. I guess no reason to expect it to break anywhere else either, but I'll update if it does.

A word of caution, the second argument, resource_name may be anything you want, and the function will return that resource as if it was a valid path.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.