In what area(s)?

/area monitoring

Describe the feature

Add logs to indicate that user-container steps into/recovers from bad status.

If a request results in container crash and restart, the request returns 502 but there's no logs/events/ways to know what had happened.

For example, follow the steps below:

1. Deploy autoscale-go sample app with 256Mi as memory resources limits.
2. Run watch kubectl get pods to keep pods status refreshed.
3. Send requests with ?bloat=1000 multiple times:
   curl -H "Host: autoscale-go.default.example.com" http://<Istio-gateway IP>?bloat=500
4. You will get response with 502 and see the pod change status into OOMKilled for a few seconds and the user-container get restarted.

There is Last State in the output of kubectl get pod -oyaml result:

Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Tue, 25 Jun 2019 17:51:04 -0700
      Finished:     Tue, 25 Jun 2019 18:08:50 -0700

whose started time is the first OOMKilled event happened and finished time is the last OOMKilled event finished(during that period the container is ready for most of the time).

There are no other logs from K8S or Knative components indicating OOMKilled happened. So unless the pod is kept alive and the operator check the correct pod, no ways to know what resulted in the 502 response.