Skip to content

Disk/image corruption with large numbers of files #2625

@tristanpemble

Description

@tristanpemble

Expected behavior

Generating large numbers of files in the container will not corrupt the files

Actual behavior

Files are corrupted

Information

Diagnose & Feedback:

Docker for Mac: version: 18.02.0-ce-mac53 (60e488931a12eeba773efb9e92be068a3c081758)
macOS: version 10.13.3 (build: 17D47)
logs: /tmp/BD6AFC3D-1322-4D56-956C-65EDDD22BE75/20180220-180841.tar.gz
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     kubernetes
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     slirp
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     menubar
[OK]     disk

Dockerfile:

FROM alpine
WORKDIR /work

# Generate 100,000 files with 10,000 alphanumeric characters each (around a gigabyte of data)
RUN for i in `seq 1 100000`; do cat /dev/urandom | tr -dc 'A-Za-z0-9' | fold -w 10000 | head -n 1 > $i.txt ; done

# Look for non-alphanumeric characters.. should output no files
RUN grep -l -e '[^A-Za-z0-9]' *.txt

Steps to reproduce the behavior

  1. First I reset Docker for Mac to factory defaults.
  2. Immediately after resetting to factory defaults (and copying diagnose & feedback information above), using the above Dockerfile, I get this output from docker build .:
    λ docker build .
    Sending build context to Docker daemon  2.048kB
    Step 1/4 : FROM alpine
    latest: Pulling from library/alpine
    ff3a5c916c92: Pull complete
    Digest: sha256:7df6db5aa61ae9480f52f0b3a06a140ab98d427f86d8d5de0bedab9b8df6b1c0
    Status: Downloaded newer image for alpine:latest
     ---> 3fd9065eaf02
    Step 2/4 : WORKDIR /work
    Removing intermediate container f0915c5abd00
     ---> 30821123326c
    Step 3/4 : RUN for i in `seq 1 100000`; do cat /dev/urandom | tr -dc 'A-Za-z0-9' | fold -w 10000 | head -n 1 > $i.txt ; done
     ---> Running in 8211bc831d57
    Removing intermediate container 8211bc831d57
     ---> 74ae74bd63f0
    Step 4/4 : RUN grep -l -e '[^A-Za-z0-9]' *.txt
     ---> Running in 0e98eafb4c78
    6390.txt
    63900.txt
    63901.txt
    63902.txt
    63903.txt
    63904.txt
    63905.txt
    63906.txt
    63907.txt
    63908.txt
    63909.txt
    6391.txt
    63910.txt
    63911.txt
    63912.txt
    63913.txt
    63914.txt
    63915.txt
    63916.txt
    63917.txt
    63918.txt
    63919.txt
    6392.txt
    63920.txt
    63921.txt
    63922.txt
    63923.txt
    63924.txt
    63925.txt
    63926.txt
    63927.txt
    63928.txt
    63929.txt
    6393.txt
    63930.txt
    63931.txt
    63932.txt
    63933.txt
    63934.txt
    63935.txt
    63936.txt
    63937.txt
    63938.txt
    63939.txt
    6394.txt
    63940.txt
    63941.txt
    63942.txt
    63943.txt
    63944.txt
    63945.txt
    63946.txt
    63947.txt
    63948.txt
    63949.txt
    6395.txt
    63950.txt
    63951.txt
    63952.txt
    63953.txt
    63954.txt
    63955.txt
    63956.txt
    63957.txt
    63958.txt
    63959.txt
    6396.txt
    63960.txt
    63961.txt
    63962.txt
    63963.txt
    63964.txt
    63965.txt
    63966.txt
    63967.txt
    63968.txt
    63969.txt
    6397.txt
    63970.txt
    63971.txt
    63972.txt
    63973.txt
    63974.txt
    63975.txt
    63976.txt
    63977.txt
    64360.txt
    64361.txt
    64362.txt
    64363.txt
    64364.txt
    64365.txt
    64366.txt
    64367.txt
    64368.txt
    64369.txt
    6437.txt
    64370.txt
    64371.txt
    64372.txt
    64373.txt
    64374.txt
    64375.txt
    64376.txt
    64377.txt
    64378.txt
    64379.txt
    6438.txt
    64380.txt
    64381.txt
    64382.txt
    64383.txt
    64384.txt
    64385.txt
    64386.txt
    64387.txt
    64388.txt
    64389.txt
    6439.txt
    64390.txt
    64391.txt
    64392.txt
    64393.txt
    64394.txt
    64395.txt
    64396.txt
    64397.txt
    64398.txt
    64399.txt
    644.txt
    6440.txt
    64400.txt
    64401.txt
    64402.txt
    64403.txt
    64404.txt
    64405.txt
    64406.txt
    64407.txt
    64408.txt
    64409.txt
    6441.txt
    64410.txt
    64411.txt
    64412.txt
    64413.txt
    64414.txt
    64415.txt
    64416.txt
    64417.txt
    64418.txt
    64419.txt
    6442.txt
    64420.txt
    64421.txt
    64422.txt
    64423.txt
    64424.txt
    64425.txt
    64426.txt
    64427.txt
    64428.txt
    64429.txt
    6443.txt
    64430.txt
    64431.txt
    64432.txt
    64433.txt
    64434.txt
    64435.txt
    64436.txt
    64437.txt
    64438.txt
    64439.txt
    6444.txt
    64440.txt
    64441.txt
    64442.txt
    64443.txt
    64444.txt
    64445.txt
    64446.txt
    64447.txt
    64448.txt
    64449.txt
    6445.txt
    64450.txt
    64451.txt
    64452.txt
    64453.txt
    64454.txt
    64455.txt
    64456.txt
    64457.txt
    64458.txt
    64459.txt
    6446.txt
    64460.txt
    64461.txt
    64462.txt
    64463.txt
    64464.txt
    64465.txt
    64466.txt
    64467.txt
    64468.txt
    64469.txt
    6447.txt
    64470.txt
    64471.txt
    64472.txt
    64473.txt
    64474.txt
    64475.txt
    64476.txt
    64477.txt
    64478.txt
    64479.txt
    6448.txt
    64480.txt
    64481.txt
    64482.txt
    64483.txt
    64484.txt
    64485.txt
    64486.txt
    64487.txt
    64488.txt
    64489.txt
    6449.txt
    64490.txt
    64491.txt
    64492.txt
    64493.txt
    64494.txt
    64495.txt
    64496.txt
    64497.txt
    64498.txt
    64499.txt
    645.txt
    6450.txt
    64500.txt
    64501.txt
    64502.txt
    64503.txt
    64504.txt
    64505.txt
    64506.txt
    64507.txt
    64508.txt
    64509.txt
    6451.txt
    64510.txt
    64511.txt
    64512.txt
    64513.txt
    64514.txt
    64515.txt
    64516.txt
    64517.txt
    64518.txt
    64519.txt
    6452.txt
    64520.txt
    64521.txt
    64522.txt
    64523.txt
    64524.txt
    64525.txt
    64526.txt
    64527.txt
    64528.txt
    64529.txt
    6453.txt
    64530.txt
    64531.txt
    64532.txt
    64533.txt
    64534.txt
    64535.txt
    64536.txt
    64537.txt
    64538.txt
    64539.txt
    6454.txt
    64540.txt
    64541.txt
    64542.txt
    64543.txt
    64544.txt
    64545.txt
    64546.txt
    64547.txt
    64548.txt
    64549.txt
    6455.txt
    64550.txt
    64551.txt
    64552.txt
    64553.txt
    64554.txt
    64555.txt
    64556.txt
    64557.txt
    64558.txt
    64559.txt
    6456.txt
    64560.txt
    64561.txt
    64562.txt
    64563.txt
    64564.txt
    64565.txt
    64566.txt
    64567.txt
    64568.txt
    64569.txt
    6457.txt
    64570.txt
    64571.txt
    64572.txt
    64573.txt
    64574.txt
    64575.txt
    64576.txt
    64577.txt
    64578.txt
    64579.txt
    6458.txt
    64580.txt
    64581.txt
    64582.txt
    64583.txt
    64584.txt
    64585.txt
    64586.txt
    64587.txt
    64588.txt
    64589.txt
    6459.txt
    64590.txt
    64591.txt
    64592.txt
    64593.txt
    64594.txt
    64595.txt
    64596.txt
    64597.txt
    64598.txt
    64599.txt
    646.txt
    6460.txt
    64600.txt
    64601.txt
    64602.txt
    64603.txt
    64604.txt
    64605.txt
    64606.txt
    64607.txt
    64608.txt
    64609.txt
    6461.txt
    64610.txt
    64611.txt
    64612.txt
    64613.txt
    64614.txt
    64615.txt
    64616.txt
    64617.txt
    64618.txt
    64619.txt
    6462.txt
    64620.txt
    64621.txt
    64622.txt
    64623.txt
    64624.txt
    64625.txt
    64626.txt
    64627.txt
    64628.txt
    64629.txt
    6463.txt
    64630.txt
    64631.txt
    64632.txt
    64633.txt
    64634.txt
    64635.txt
    64636.txt
    64637.txt
    64638.txt
    64639.txt
    6464.txt
    64640.txt
    64641.txt
    64642.txt
    64643.txt
    64644.txt
    64645.txt
    64646.txt
    64647.txt
    64648.txt
    64649.txt
    6465.txt
    64650.txt
    64651.txt
    64652.txt
    64653.txt
    64654.txt
    64655.txt
    64656.txt
    64657.txt
    64658.txt
    64659.txt
    6466.txt
    64660.txt
    64661.txt
    64662.txt
    64663.txt
    64664.txt
    64665.txt
    64666.txt
    64667.txt
    64668.txt
    64669.txt
    6467.txt
    64670.txt
    64671.txt
    64672.txt
    64673.txt
    64674.txt
    64675.txt
    64676.txt
    64677.txt
    64678.txt
    64679.txt
    6468.txt
    64680.txt
    64681.txt
    64682.txt
    64683.txt
    64684.txt
    64685.txt
    64686.txt
    64687.txt
    64688.txt
    64689.txt
    6469.txt
    64690.txt
    64691.txt
    64692.txt
    64693.txt
    64694.txt
    64695.txt
    64696.txt
    64697.txt
    64698.txt
    64699.txt
    647.txt
    6470.txt
    64700.txt
    64701.txt
    64702.txt
    64703.txt
    64704.txt
    64705.txt
    64706.txt
    64707.txt
    64708.txt
    64709.txt
    6471.txt
    64710.txt
    64711.txt
    64712.txt
    64713.txt
    64714.txt
    64715.txt
    64716.txt
    64717.txt
    64718.txt
    64719.txt
    6472.txt
    64720.txt
    64721.txt
    64722.txt
    64723.txt
    64724.txt
    64725.txt
    64726.txt
    64727.txt
    64728.txt
    64729.txt
    6473.txt
    64730.txt
    64731.txt
    64732.txt
    64733.txt
    64734.txt
    64735.txt
    64736.txt
    64737.txt
    64738.txt
    64739.txt
    6474.txt
    64740.txt
    64741.txt
    64742.txt
    64743.txt
    64744.txt
    64745.txt
    64746.txt
    64747.txt
    64748.txt
    64749.txt
    6475.txt
    64750.txt
    64751.txt
    64752.txt
    64753.txt
    64754.txt
    64755.txt
    64756.txt
    64757.txt
    64758.txt
    64759.txt
    6476.txt
    64760.txt
    64761.txt
    64762.txt
    64763.txt
    64764.txt
    64765.txt
    64766.txt
    64767.txt
    64768.txt
    64769.txt
    6477.txt
    64770.txt
    64771.txt
    64772.txt
    64773.txt
    64774.txt
    64775.txt
    64776.txt
    64777.txt
    64778.txt
    64779.txt
    6478.txt
    64780.txt
    64781.txt
    64782.txt
    64783.txt
    64784.txt
    64785.txt
    64786.txt
    64787.txt
    64788.txt
    64789.txt
    6479.txt
    64790.txt
    64791.txt
    64792.txt
    64793.txt
    64794.txt
    64795.txt
    64796.txt
    64797.txt
    64798.txt
    64799.txt
    648.txt
    6480.txt
    64800.txt
    64801.txt
    64802.txt
    64803.txt
    64804.txt
    64805.txt
    64806.txt
    64807.txt
    64808.txt
    64809.txt
    6481.txt
    64810.txt
    64811.txt
    64812.txt
    64813.txt
    64814.txt
    64815.txt
    64816.txt
    64817.txt
    64818.txt
    64819.txt
    6482.txt
    64820.txt
    64821.txt
    Removing intermediate container 0e98eafb4c78
     ---> 9968c5d7fc72
    Successfully built 9968c5d7fc72
    
  3. Examine one of the files in the list with docker run --rm 9968c5d7fc72 cat 6482.txt (truncated output):
     λ docker run --rm 9968c5d7fc72 cat 6482.txt
     �wa\�X�E=IΪm�SP�	��-F�#ɕaQ1�A��R#�*9P�#�j���z�X�2��be�ԵG}Jc#�.�-���kC6�ЗE���N��A�,���1:��R�������R�?A}B��:�QK��GU�5'F�^]cQ�S�Y��9��D�L�r9��P4�#�
     ��y�(f>h��=Hҍs"o{�?\���Y�T���:%������A;��H��% �g� 8w�\�1)�?k�v-0֪�k�;������O����4B%И�FA�	��`�IL�s!��L��h���8ِ���;aV�+��mXi� �vD�g�K~���݆�#l���'�Q��IS|�&Xi�
                                       s�	����/���v�Dj�����F�}B�����%򡏱���{Dfg=�į�~��I���|L[)�I�

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions