Add an mtime file watcher.

[dlorenc]

No description provided.

[dlorenc]

This can help us work around the issues caused by inotify-style watchers.

[dlorenc]

TODO:

• Figure out the best way to configure which watcher to use
• Tests
• benchmarking

[ajbouh]

In other projects, I've been very satisfied with using https://facebook.github.io/watchman/

It's a separate service that's optimized for efficient file watching on major operating systems. It supports features like a logical clock and providing a digest of fes when they change. There's a json or binary protocol that's accessible via a Unix socket.

[dgageot]

@dlorenc @r2d4 I think we could get rid of the inotify watcher altogether because it won't go any further. Changing the list of dependencies based on file changes with inotify is too tricky

[dlorenc]

@dlorenc @r2d4 I think we could get rid of the inotify watcher altogether because it won't go any further. Changing the list of dependencies based on file changes with inotify is too tricky

SGTM, how about an environment variable to switch between the two for now in-case there is a large perf hit for some workloads? We can switch the default to mtime in this PR but leave inotify until we stop hearing complaints about mtime.

[dlorenc]

An env var would be easier than a flag because we can remove it without breaking people that pass it on the cmd line.

[dlorenc]

Dropping the WIP, this should be ready for a look.

I prioritized ease-of-removal of fsnotify rather than doing a full refactor.

[dlorenc]

These tests are passing for me but failing in travis, i'll keep debugging:

--- PASS: TestWatch (5.03s)
    --- PASS: TestWatch/watch_unknown_file_mtime (0.00s)
    --- PASS: TestWatch/write_files_mtime (2.01s)
    --- PASS: TestWatch/ignore_file_mtime (2.00s)
    --- PASS: TestWatch/watch_unknown_file_fsnotify (0.00s)
    --- PASS: TestWatch/write_files_fsnotify (0.51s)
    --- PASS: TestWatch/ignore_file_fsnotify (0.51s)

[dlorenc]

Tests passing now.

[r2d4]

I think we should do some testing with minikube/d4d to make sure this works well. We still have the clock skew issues in minikube.

[dlorenc]

I think we should do some testing with minikube/d4d to make sure this works well. We still have the clock skew issues in minikube.

The skew isn't important, we take an initial snapshot then just check to see if the values have changed at all from that.

What exactly do you mean by testing in d4d? Watch happens on the local machine, not in the build step itself.

[dgageot]

@dlorenc I'm sorry you have to rebase the code.

[dlorenc]

@dgageot anything blocking this?

[dgageot]

Absolutely not! Let me merge it
LGTM