Popen used in a way that may cause deadlock

[ekoyle]

I noticed a minor issue here:

package-manager/zeekpkg/manager.py

Lines 2748 to 2786 in da77d8f

	build = subprocess.Popen(
	build_command,
	shell=True,
	cwd=clone.working_dir,
	env=env,
	bufsize=bufsize,
	stdout=subprocess.PIPE,
	stderr=subprocess.PIPE,
	)
	try:
	buildlog = self.package_build_log(clone.working_dir)
	with open(buildlog, "wb") as f:
	LOG.info(
	'installing "%s": writing build log: %s',
	package,
	buildlog,
	)
	f.write("=== STDERR ===\n".encode(std_encoding(sys.stderr)))
	while True:
	data = build.stderr.read(bufsize)
	if data:
	f.write(data)
	else:
	break
	f.write("=== STDOUT ===\n".encode(std_encoding(sys.stdout)))
	while True:
	data = build.stdout.read(bufsize)
	if data:
	f.write(data)
	else:
	break

This pattern will cause a deadlock if the child process writes enough to stdout to fill the buffer and block.

p = subprocess.Popen(..., stdout=subrpocess.PIPE, stderr=subprocess.PIPE)

# read p.sdterr until EOF
while True:
    # this read() may cause a deadlock as the stdout pipe could fill, causing the child process
    # to block forever on a write to stdout while the parent process is blocking on this read
    # and therefore not reading from p.stdout
    data = p.stderr.read(bufsize)

    if data:
        f.write(data)
    else:
        break

# read p.stdout until EOF
...

It is recommended to use Popen.communicate() to safely handle cases where subprocess.PIPE is used in a Popen (especially more than once) to avoid this issue. If you must read directly from more than one pipe, then epoll(), select(), or similar should be used to avoid blocking reads which could result in a deadlock.