In kubernetes, if I desired to have pods for a specific task not always required is it possible to have the system spin pods up for a given task when needed? Can this be managed through the backend code (in my instance python)? Do services need to be defined in a certain way in advance?
I found an api for docker management in Python, would this work for Kubernetes?
If i have understood correctly you want to run the PODs when required not all the time.
in that case you can use the python client library of Kubernetes and manage the PODs or replicas of deployment.
you can deploy your service with replica count zero and once required you can scale up the pods using the client to as much required.
Kubernetes python client: https://github.com/kubernetes-client/python
if your service is already running you can use this API :
GET /apis/extensions/v1beta1/namespaces/{namespace}/deployments/{name}/scale
here using just curl also you can spin up the replicas
API_URL="http://kubernetes:8080/apis/extensions/v1beta1/namespaces/{namespace}/deployments/{name}/scale"
curl -H 'Accept: application/json' $API_URL > scale.json
# edit scale.json
curl -X PUT -d@scale.json -H 'Content-Type: application/json' $API_URL
I don't know exactly what you want to achieve because you are not giving enough information. Here are some tools you migh want to check:
keda - event driven autoscaler, can scale down to zero
k8s jobs - runs one time jobs
knative - can scale down to zero
You can also just use k8s python client library to run a pod, make it do whatever you want and then delete it.