Question

0

224

Views

How do you iteratively develop with docker?

How does one iteratively develop their app using Docker? I have only just started using it and my workflow is very slow, so I'm pretty sure I'm using it wrong.

I'm following along with a python machine learning course on Youtube, and so I am using Docker to work with python 3. I know I can use virtualenv or a VM, but I want to learn Docker as well so bear with me.

My root directory looks like so:

Dockerfile  main.py*

My docker file:

FROM python
COPY . /src
RUN pip install quandl
RUN pip install pandas
CMD ["python", "/src/main.py"]

And the Python file:

#!/usr/bin/env python

import pandas as pd
import quandl

print("Hello world from main.py")

df = quandl.get("WIKI/GOOGL")

print("getting data frame for WIKI/GOOGL")
print(df.head())

My workflow has been:

Learn something new from the tutorial
Update python file
Build the docker image: docker build -t myapp .
Run the app: docker run my app python /src/main.py

Questions:

How can I speed this all up? For every change I want to try, I end up rebuilding. This causes pip to get dependencies each time which takes way too long.
Instead of editing a python file and running it, how might a get an interactive shell from the python version running in the container?
If I wanted my program to write out a file, how could I get this file back to my local system from the container after the program has finished?

Thanks for the help!

Edit: I should add, this was the tutorial I was following in general to run some python code in Docker: https://www.civisanalytics.com/blog/using-docker-to-run-python/

about 4 years ago · Santiago Trujillo

1 answers

Answer question

0

Answer question

Find remote jobs

Accepted Answer · 2022-06-13T19:20:39.888Z

Speeding up the rebuild process

The simplest thing you can do is reorder your Dockerfile.

FROM python
RUN pip install quandl
RUN pip install pandas
COPY . /src
CMD ["python", "/src/main.py"]

The reason this helps is that Docker will re-use the cached build for commands it has already run. Now when you rebuild after modifying your source code, it will re-use the build results for the pip commands, as they do not need to be run again. It will only run the COPY step.

Getting a python shell

You can exec a shell in the running container and run your python command.

docker exec -it <container-id> bash
python <...>

Or, you can run a container with just a shell, and skip running your app entirely (then, run it however you want).

docker run -it <image> bash
python <...>

Writing outside the container

Mount an external directory into the container. Then write to the mounted path.

docker run -v /local/path:/path <.. rest of command ..>

Then when you write in the container to /path/file, the file will show up outside the container at /local/path/file.

0

224