Why does my Python TensorFlow 2.x model execution not use eager mode by API calls?

I am trying to set TensorFlow to eager mode as my model is built to run in eager mode (rather than graph mode).

I traced down that some percent of my inference requests land in non-eager mode and those requests fail. It is a complete mystery why it is happening for say 10% of requests.

I added a code block to do a sanity check and am trying to force TF to run in eager mode if it is not. Oddly enough, the code continued to fail as TF goes into graph mode.

Now, I added some logs to see if the TF is going into eager mode. The boolean comes back as False in both cases.

The code is here:

        if not tf.executing_eagerly():            logger.info("Eager execution is not enabled. Enabling eager execution.", kv=self.stats_tags)            tf.compat.v1.enable_eager_execution()            logger.info(f"Eager execution mode is enabled: {tf.executing_eagerly()}", kv=self.stats_tags)            tf.config.experimental_run_functions_eagerly(True)            logger.info(f"Eager execution mode is enabled: {tf.executing_eagerly()}", kv=self.stats_tags)

Any ideas for what is going on? Why?

I expect that one of the API calls above will put TensorFlow in eager mode.