莫非 - 技术博客
0%

在Python中禁止fbprophet打印日志

发表于 更新于 分类于
使用fbprophet会打印很多无用的日志，如何禁止打印这些日志呢？

背景

在使用fbprophet库的Prophet.fit()时，会打印很多日志，格式如下：

1
2
3
4
5
6
7
8
9
10
11
Initial log joint probability = -7.89676
Iteration  1. Log joint probability =  -0.553355. Improved by 7.3434.
Iteration  2. Log joint probability =   0.562761. Improved by 1.11612.
Iteration  3. Log joint probability =    2.35633. Improved by 1.79356.
Iteration  4. Log joint probability =    3.01258. Improved by 0.65625.
Iteration  5. Log joint probability =    4.58069. Improved by 1.56812.
Iteration  6. Log joint probability =    7.33663. Improved by 2.75594.
Iteration  7. Log joint probability =    7.85717. Improved by 0.520535.
Iteration  8. Log joint probability =    9.81286. Improved by 1.95569.
Iteration  9. Log joint probability =    11.0328. Improved by 1.21995.
Iteration 10. Log joint probability =    12.2614. Improved by 1.22857.

在通过日志排查错误时，大量的无用日志加大了排查错误的难度，所以需要禁止打印这些日志。

查找源代码

在vscode中使用关键字搜索，在python3.6/site-packages/pystan/stan/src/stan/services/optimize/newton.hpp中找到打印日志的源代码，内容如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
namespace stan {
  namespace services {
    namespace optimize {
      template <class Model>
      int newton(Model& model, stan::io::var_context& init,
                 unsigned int random_seed, unsigned int chain,
                 double init_radius, int num_iterations,
                 bool save_iterations,
                 callbacks::interrupt& interrupt,
                 callbacks::logger& logger,
                 callbacks::writer& init_writer,
                 callbacks::writer& parameter_writer) {
        ......

        std::stringstream msg;
        msg << "Initial log joint probability = " << lp;
        logger.info(msg);

        ......

        double lastlp = lp;
        for (int m = 0; m < num_iterations; m++) {
          ......

          std::stringstream msg2;
          msg2 << "Iteration "
               << std::setw(2) << (m + 1) << "."
               << " Log joint probability = " << std::setw(10) << lp
               << ". Improved by " << (lp - lastlp) << ".";
          logger.info(msg2);

          ......
        }

        ......
        return error_codes::OK;
      }

    }
  }
}
#endif

方案一：修改日志级别（失败）

尝试修改fbprophet的日志级别来禁止打印

1
2
3
4
5
6
7
8
9
10
11
12
13
import logging
from fbprophet import Prophet
import pandas as pd

logging.getLogger('fbprophet').setLevel(logging.WARNING)

data = [['2021-01-26', 1], ['2021-01-27', 2]]
df = pd.DataFrame(data, columns=['ds', 'y'])
df['cap']= 10

m = Prophet(growth='logistic')
logging.basicConfig(level=logging.WARNING)
m.fit(df)

测试发现这种方法没有作用。

方案二：使用禁止输出的上下文管理器（成功）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
class suppress_stdout_stderr(object):
    '''
    A context manager for doing a "deep suppression" of stdout and stderr in
    Python, i.e. will suppress all print, even if the print originates in a
    compiled C/Fortran sub-function.
       This will not suppress raised exceptions, since exceptions are printed
    to stderr just before a script exits, and after the context manager has
    exited (at least, I think that is why it lets exceptions through).
    '''
    def __init__(self):
        # Open a pair of null files
        self.null_fds = [os.open(os.devnull, os.O_RDWR) for x in range(2)]
        # Save the actual stdout (1) and stderr (2) file descriptors.
        self.save_fds = [os.dup(1), os.dup(2)]
    def __enter__(self):
        # Assign the null pointers to stdout and stderr.
        os.dup2(self.null_fds[0], 1)
        os.dup2(self.null_fds[1], 2)
    def __exit__(self, *_):
        # Re-assign the real stdout/stderr back to (1) and (2)
        os.dup2(self.save_fds[0], 1)
        os.dup2(self.save_fds[1], 2)
        # Close the null files
        for fd in self.null_fds + self.save_fds:
            os.close(fd)

import logging
from fbprophet import Prophet
import pandas as pd

data = [['2021-01-26', 1], ['2021-01-27', 2]]
df = pd.DataFrame(data, columns=['ds', 'y'])
df['cap']= 10

m = Prophet(growth='logistic')
with suppress_stdout_stderr():
    m.fit(df)

这种方法可以禁止日志打印。

参考文档

https://github.com/facebook/prophet/issues/223