I have a Laravel app deployed on Azure Kubernetes with docker. My app is having trouble uploading files whenever I try to upload a file I get this error from Laravel (showing only a part of the stacktrace):
chmod(): Operation not permitted {"userId":1,"exception":"[object] (ErrorException(code: 0): chmod(): Operation not permitted at /var/www/my-app/vendor/league/flysystem/src/Adapter/Local.php:367) [stacktrace]
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'chmod(): Operat...', '/var/www/my-app...', 367, Array)
#1 /var/www/my-app/vendor/league/flysystem/src/Adapter/Local.php(367): chmod('/var/www/my-app...', 420)
However in my Dockerfile I have 'chowned' the storage folder:
RUN chown -R www-data:www-data \
/var/www/my-app/storage \
/var/www/my-app/bootstrap/cache
I should mention that the file gets uploaded but fails to continue with rest of the code because of the chmod exception.
In an attempt to debug the issue I ran kubectl exec to get a shell into the pod, by default it logs in as root. I cd to the uploaded files and try changing the permissions as root by running chmod 420 nameOfFile.ext and that works, so I change it back to permissions 777. However since Laravel is using the apache user "www-data", I run su www-data -s /bin/bash then try to change the permission of the same file by running chmod 420 nameOfFile.ext and I get this error:
chmod: changing permissions of "nameOfFile.ext" Operation not permitted
So that left me to wonder if the '-R' in chown only worked on files and folders that were directly a sub file or folder. So I switched back to root user 'chowned' the folder where the files were directly in, then switched back to www-data user and tried running chmod on the file but still got the same error.
[EDIT] I also should mention the application is using Azure file service as the persistent volume. Would changing it to a blob service help?
[EDIT] This is what my complete Dockerfile looks like: https://pastebin.com/zLSyfqK8
I've been on this issue for a while, any help is appreciated. Let me know if you need any other necessary info.
I also should mention the application is using Azure file service as the persistent volume.
That's the problem. Azure Files provides a CIFS share, which is mounted under Linux as a cifs mount. CIFS shares do not provide UNIX type file permissions and UNIX type uid/gid storage.
When mounting the CIFS share, the Linux machine uses a username/password to authenticate at the CIFS server. Every access through this mount will actually use this username on the CIFS server, regardless of which UNIX user is initiating the file system operations. According to this GitHub issue, Azure Files does not support UNIX extensions, and that means that the Linux client would have to emulate UNIX uid/gid & permissions. The server does not store or supply these parameters. So at mount time, you have to add uid, gid, file_mode and dir_mode parameters to set these data.
If you are using OS level CIFS mounts (through /etc/fstab), or manual CLI mount commands, you have to add these options to the mount command (see man mount.cifs).
Eg: mount -t cifs -o file_mode=0644,dir_mode=0755,uid=80,gid=80 ...
If you are using Kubernetes volumes, you have to add these options to the volume parameters (see Azure Files share in Kubernetes).
Eg:
apiVersion: v1
kind: PersistentVolume
metadata:
name: azurefile
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
storageClassName: azurefile
azureFile:
secretName: azure-secret
shareName: aksshare
readOnly: false
mountOptions:
- dir_mode=0755
- file_mode=0644
- uid=80
- gid=80
- mfsymlinks