タグ: datadog

投稿日:

datadog-agentを動かすと大量のエラーが出ていたので直してみた

centos 7.3 にdatadogを導入してしばらくたっていましたが、ログを見ると大量のログが..
/etc/init.d/datadog-agent info でみると v 5.13.0 のようです

/var/log/message に

dd.collector[19776]: WARNING (disk.py:106): Unable to get disk metrics for /var/lib/docker/overlay/73186596b1b91e9e43aa5bfba55d023ed85e1108a25cf58360fa4fd7bb620c81/merged: [Errno 13] Permission denied: '/var/lib/docker/overlay/73186596b1b91e9e43aa5bfba55d023ed85e1108a25cf58360fa4fd7bb620c81/merged'
dd.collector[19776]: WARNING (disk.py:106): Unable to get disk metrics for /var/lib/docker/containers/33c87f172c8ae018545702ea12a8d75202c7b7e9aefcfc6a567abb419f09e26b/shm: [Errno 13] Permission denied: '/var/lib/docker/containers/33c87f172c8ae018545702ea12a8d75202c7b7e9aefcfc6a567abb419f09e26b/shm'
dd.collector[19776]: WARNING (disk.py:106): Unable to get disk metrics for net:[4026532202]: [Errno 2] No such file or directory: 'net:[4026532202]'

の３種類のログがたくさん出てました

dd.collector[1724]: WARNING (disk.py:109): Unable to get disk metrics for ... · Issue #2932 · DataDog/dd-agent

を参考に（issueは閉じられていませんが）

DISK='/my/mountpoint' /opt/datadog-agent/embedded/bin/python -c 'import psutil; import os; print [part.fstype for part in psutil.disk_partitions(all=True) if part.mountpoint == os.environ["DISK"]][0]'

のpythonスクリプトでエラーログの発生箇所を調べると

$ DISK='/var/lib/docker/overlay/73186596b1b91e9e43aa5bfba55d023ed85e1108a25cf58360fa4fd7bb620c81/merged' /opt/datadog-agent/embedded/bin/python -c 'import psutil; import os; print [part.fstype for part in psutil.disk_partitions(all=True) if part.mountpoint == os.environ["DISK"]][0]'
>  overlay

$ DISK='/var/lib/docker/containers/33c87f172c8ae018545702ea12a8d75202c7b7e9aefcfc6a567abb419f09e26b/shm' /opt/datadog-agent/embedded/bin/python -c 'import psutil; import os; print [part.fstype for part in psutil.disk_partitions(all=True) if part.mountpoint == os.environ["DISK"]][0]'
>  tmpfs

$ DISK='net:[4026532202]' /opt/datadog-agent/embedded/bin/python -c 'import psutil; import os; print [part.fstype for part in psutil.disk_partitions(all=True) if part.mountpoint == os.environ["DISK"]][0]'
>  proc

/etc/dd-agent/disk.yml.default をコピーして /etc/dd-agent/disk.yml を作成
スクリプトで出力されたものを excluded_filesystems に指定

   excluded_filesystems:
      - overlay
      - tmpfs
      - proc

これでdd-agentをリロード

$ sudo /etc/init.d/dd-agent reload

エラーログには出なくなりました