I had an intention to dynamically generate a robots.txt file for different domain names.

I removed the file from /web/ directory. I added

bits_cb.common:
    resource: "@BitsClientoboxBundle/Controller/CommonController.php"
    type: annotation

to routing.yml.

I added

/**
 * @Route("/robots.txt", name="robots")
 * @param Request $request
 */
public function indexAction(Request $request)
{
 //
}

to CommonController.php.

The debug:router console command there is route:

robots    ANY    ANY    ANY    /robots.txt

But if I try to get from http://{{sitename}}/robots.txt it returns 404. To be sure, if I change

* @Route("/robots.txt", name="robots")

to

* @Route("/robots", name="robots")

and get from http://{{sitename}}/robots it works okay.