Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PyCall build error owing to JSONDecodeError #1035

Open
SimonEnsemble opened this issue May 9, 2023 · 1 comment
Open

PyCall build error owing to JSONDecodeError #1035

SimonEnsemble opened this issue May 9, 2023 · 1 comment

Comments

@SimonEnsemble
Copy link

was working a few days ago, but after an up, build PyCall fails, with the following build log.

Collecting package metadata (current_repodata.json): ...working... failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/exceptions.py", line 1132, in __call__
        return func(*args, **kwargs)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/cli/main.py", line 69, in main_subshell
        exit_code = do_call(args, p)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/cli/conda_argparse.py", line 122, in do_call
        return getattr(module, func_name)(args, parser)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/notices/core.py", line 121, in wrapper
        return func(*args, **kwargs)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/cli/main_install.py", line 20, in execute
        install(args, parser, 'install')
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/cli/install.py", line 264, in install
        unlink_link_transaction = solver.solve_for_transaction(
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/solve.py", line 134, in solve_for_transaction
        unlink_precs, link_precs = self.solve_for_diff(update_modifier, deps_modifier,
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/solve.py", line 177, in solve_for_diff
        final_precs = self.solve_final_state(update_modifier, deps_modifier, prune, ignore_pinned,
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/solve.py", line 282, in solve_final_state
        ssc = self._collect_all_metadata(ssc)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/common/io.py", line 84, in decorated
        return f(*args, **kwds)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/solve.py", line 449, in _collect_all_metadata
        index, r = self._prepare(prepared_specs)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/solve.py", line 1064, in _prepare
        reduced_index = get_reduced_index(self.prefix, self.channels,
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/index.py", line 267, in get_reduced_index
        new_records = SubdirData.query_all(spec, channels=channels, subdirs=subdirs,
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/subdir_data.py", line 169, in query_all
        result = tuple(chain.from_iterable(executor.map(subdir_query, channel_urls)))
      File "/Users/cokes/.julia/conda/3/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
        yield _result_or_cancel(fs.pop())
      File "/Users/cokes/.julia/conda/3/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
        return fut.result(timeout)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/concurrent/futures/_base.py", line 451, in result
        return self.__get_result()
      File "/Users/cokes/.julia/conda/3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
        raise self._exception
      File "/Users/cokes/.julia/conda/3/lib/python3.10/concurrent/futures/thread.py", line 58, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/subdir_data.py", line 158, in subdir_query
        return tuple(
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/subdir_data.py", line 174, in query
        self.load()
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/subdir_data.py", line 264, in load
        _internal_state = self._load()
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/core/subdir_data.py", line 315, in _load
        cache.load_state()  # XXX should this succeed even if FileNotFound?
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 570, in load_state
        self.load(state_only=True)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 528, in load
        state = json.loads(state_file.read())
      File "/Users/cokes/.julia/conda/3/lib/python3.10/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
      File "/Users/cokes/.julia/conda/3/lib/python3.10/json/decoder.py", line 340, in decode
        raise JSONDecodeError("Extra data", s, end)
    json.decoder.JSONDecodeError: Extra data: line 9 column 2 (char 289)

`$ /Users/cokes/.julia/conda/3/bin/conda install -y numpy`

  environment variables:
                 CIO_TEST=<not set>
                  CONDARC=/Users/cokes/.julia/conda/3/condarc-julia.yml
             CONDA_PREFIX=/Users/cokes/.julia/conda/3
               CONDA_ROOT=/Users/cokes/.julia/conda/3
           CURL_CA_BUNDLE=<not set>
                 GEM_PATH=/Users/cokes/.gem/ruby/3.1.3:/Users/cokes/.rubies/ruby-
                          3.1.3/lib/ruby/gems/3.1.0
                 INFOPATH=/opt/homebrew/share/info:
          JULIA_LOAD_PATH=@:/var/folders/b3/p558slm10qj7tzmkx22rhyc80000gn/T/jl_YR0kPl
               LD_PRELOAD=<not set>
                  MANPATH=/opt/homebrew/share/man::
                     PATH=/Users/cokes/.gem/ruby/3.1.3/bin:/Users/cokes/.rubies/ruby-
                          3.1.3/lib/ruby/gems/3.1.0/bin:/Users/cokes/.rubies/ruby-
                          3.1.3/bin:/Users/cokes/opt/anaconda3/bin:/Users/cokes/opt/anaconda3/co
                          ndabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cry
                          ptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/
                          usr/local/go/bin:/var/run/com.apple.security.cryptexd/codex.system/boo
                          tstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system
                          /bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/b
                          ootstrap/usr/appleinternal/bin
         PYTHONIOENCODING=UTF-8
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : base
    active env location : /Users/cokes/.julia/conda/3
       user config file : /Users/cokes/.condarc
 populated config files : /Users/cokes/.julia/conda/3/.condarc
                          /Users/cokes/.condarc
                          /Users/cokes/.julia/conda/3/condarc-julia.yml
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.10.8.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.3.1=0
                          __unix=0=0
       base environment : /Users/cokes/.julia/conda/3  (writable)
      conda av data dir : /Users/cokes/.julia/conda/3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/cokes/.julia/conda/3/pkgs
                          /Users/cokes/.conda/pkgs
       envs directories : /Users/cokes/.julia/conda/3/envs
                          /Users/cokes/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.3.1 requests/2.28.1 CPython/3.10.8 Darwin/22.4.0 OSX/13.3.1
                UID:GID : 501:20
             netrc file : None
           offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

Upload successful.
┌ Info: Using the Python distribution in the Conda package by default.
└ To use a different Python version, set ENV["PYTHON"]="pythoncommand" and re-run Pkg.build("PyCall").
[ Info: Running `conda install -y numpy` in root environment
ERROR: LoadError: failed process: Process(setenv(`/Users/cokes/.julia/conda/3/bin/conda install -y numpy`,["_CE_M=", "PATH=/Users/cokes/.gem/ruby/3.1.3/bin:/Users/cokes/.rubies/ruby-3.1.3/lib/ruby/gems/3.1.0/bin:/Users/cokes/.rubies/ruby-3.1.3/bin:/Users/cokes/opt/anaconda3/bin:/Users/cokes/opt/anaconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin", "MANPATH=/opt/homebrew/share/man::", "ITERM_PROFILE=Default", "USER=cokes", "RUBY_ENGINE=ruby", "RUBYOPT=", "SHELL=/bin/zsh", "LC_TERMINAL_VERSION=3.4.19", "COLORFGBG=7;0"  …  "SECURITYSESSIONID=186a3", "TERM_PROGRAM_VERSION=3.4.19", "COMMAND_MODE=unix2003", "PWD=/Users/cokes/.julia/packages", "RUBY_ROOT=/Users/cokes/.rubies/ruby-3.1.3", "TERM_PROGRAM=iTerm.app", "GEM_HOME=/Users/cokes/.gem/ruby/3.1.3", "CONDA_PREFIX=/Users/cokes/.julia/conda/3", "TERM_SESSION_ID=w0t7p0:B4D22DC8-2AB8-4DAA-96B7-7F7B563EE985", "JULIA_LOAD_PATH=@:/var/folders/b3/p558slm10qj7tzmkx22rhyc80000gn/T/jl_YR0kPl"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run
   @ ./process.jl:477 [inlined]
 [4] runconda(args::Cmd, env::String)
   @ Conda ~/.julia/packages/Conda/kOnIE/src/Conda.jl:128
 [5] add(pkg::String, env::String; channel::String)
   @ Conda ~/.julia/packages/Conda/kOnIE/src/Conda.jl:224
 [6] add (repeats 2 times)
   @ ~/.julia/packages/Conda/kOnIE/src/Conda.jl:222 [inlined]
 [7] top-level scope
   @ ~/.julia/packages/PyCall/twYvK/deps/build.jl:79
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [9] top-level scope
   @ none:5
in expression starting at /Users/cokes/.julia/packages/PyCall/twYvK/deps/build.jl:43

version info

  • PyCall v1.95.1
  • Conda v1.8.0
julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.5.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores
@SimonEnsemble
Copy link
Author

SimonEnsemble commented May 9, 2023 •
edited
Loading

oddly, setting ENV["PYTHON"] = "/Users/cokes/opt/anaconda3/bin/python" (output of which python), then build PyCall fails too. 👀 '

UPDATE: solved by (i) uninstalling anaconda, (ii) reinstalling proper anaconda for my M1 arch. 😄 above is still an issue tho, but using my own Python installation and in business now! 🚀

ERROR: Error building `PyCall`:
DEBUG is_windows = False
DEBUG is_apple = True
DEBUG Candidate: None
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/python3.9/config-3.9-darwin/libpython3.9.a
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/python3.9/config-3.9-darwin/libpython3.9.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/python3.9/config-3.9-darwin/libpython3.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/python3.9/config-3.9-darwin/libpython.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/libpython3.9.a
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/libpython3.9.dylib
DEBUG Found: /Users/cokes/opt/anaconda3/lib/libpython3.9.dylib
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/libpython3.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/lib/libpython.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/libpython3.9.a
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/libpython3.9.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/libpython3.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/libpython.dylib
DEBUG Not found.
DEBUG Candidate: /Users/cokes/opt/anaconda3/bin/../lib/libpython3.9.dylib
DEBUG Found: /Users/cokes/opt/anaconda3/lib/libpython3.9.dylib
dlopen(/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib) ==> ErrorException("could not load library \"/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib\"\ndlopen(/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib, 0x0009): tried: '/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib' (no such file), '/Users/cokes/opt/anaconda3/lib/libpython3.9.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))")
dlopen(libpython3.9) ==> ErrorException("could not load library \"libpython3.9\"\ndlopen(libpython3.9.dylib, 0x0009): tried: 'libpython3.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpython3.9.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libpython3.9.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libpython3.9.dylib' (no such file), '/usr/lib/libpython3.9.dylib' (no such file, not in dyld cache), 'libpython3.9.dylib' (no such file), '/usr/local/lib/libpython3.9.dylib' (no such file), '/usr/lib/libpython3.9.dylib' (no such file, not in dyld cache)")
dlopen(libpython3.9) ==> ErrorException("could not load library \"libpython3.9\"\ndlopen(libpython3.9.dylib, 0x0009): tried: 'libpython3.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpython3.9.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libpython3.9.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libpython3.9.dylib' (no such file), '/usr/lib/libpython3.9.dylib' (no such file, not in dyld cache), 'libpython3.9.dylib' (no such file), '/usr/local/lib/libpython3.9.dylib' (no such file), '/usr/lib/libpython3.9.dylib' (no such file, not in dyld cache)")
dlopen(libpython3) ==> ErrorException("could not load library \"libpython3\"\ndlopen(libpython3.dylib, 0x0009): tried: 'libpython3.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpython3.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libpython3.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libpython3.dylib' (no such file), '/usr/lib/libpython3.dylib' (no such file, not in dyld cache), 'libpython3.dylib' (no such file), '/usr/local/lib/libpython3.dylib' (no such file), '/usr/lib/libpython3.dylib' (no such file, not in dyld cache)")
dlopen(libpython) ==> ErrorException("could not load library \"libpython\"\ndlopen(libpython.dylib, 0x0009): tried: 'libpython.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpython.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libpython.dylib' (no such file), '/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libpython.dylib' (no such file), '/usr/lib/libpython.dylib' (no such file, not in dyld cache), 'libpython.dylib' (no such file), '/usr/local/lib/libpython.dylib' (no such file), '/usr/lib/libpython.dylib' (no such file, not in dyld cache)")
ERROR: LoadError: Couldn't find libpython; check your PYTHON environment variable.

The python executable we tried was /Users/cokes/opt/anaconda3/bin/python (= version 3.9).
Re-building with
    ENV["PYCALL_DEBUG_BUILD"] = "yes"
may provide extra information for why it failed.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] find_libpython(python::String; _dlopen::typeof(Base.Libc.Libdl.dlopen))
   @ Main ~/.julia/packages/PyCall/twYvK/deps/buildutils.jl:72
 [3] find_libpython(python::String)
   @ Main ~/.julia/packages/PyCall/twYvK/deps/buildutils.jl:40
 [4] top-level scope
   @ ~/.julia/packages/PyCall/twYvK/deps/build.jl:82
 [5] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [6] top-level scope
   @ none:5
in expression starting at /Users/cokes/.julia/packages/PyCall/twYvK/deps/build.jl:43

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@SimonEnsemble