Updated answer for the updated question:

Yes, you can apply it to a single 'Ingress' by using the nginx.ingress.kubernetes.io/whitelist-source-range annotation. For example:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-myservice
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/whitelist-source-range: "CIDR1,CIDR2,CIDR3"
spec:
  rules:
...

Original answer for the original question:

Yes, you can. Essentially, the 'Ingresses' will use the same ingress controller as long as they have the ingress controller annotation. For example, for an Nginx ingress controller, an Ingress would look something like this:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-myservice
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
...

Then on the ConfigMap you can add something like this:

data:
  ...
  whitelist-source-range: "CIDR1,CIDR2,CIDR3"
  ...

    whitelist-source-range: >- 
      "{{- $.Values.ingress.CIDR1 }},
       {{- $.Values.ingress.CIDR1 }},
       {{- $.Values.ingress.CIDR3 }}"

with values.yaml

ingress:
  enabled: true
  annotations: {}
  hosts:
    - host: chart-example.local
      paths: []
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

  CIDR1: A.B.C.D/32,A.B.C.D/32
  CIDR2: A.B.C.D/32,A.B.C.D/32
  CIDR3: A.B.C.D/32,A.B.C.D/32