Navigation

    Quantiacs Community

    • Register
    • Login
    • Search
    • Categories
    • News
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    External Libraries

    Support
    5
    8
    644
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      antinomy last edited by

      Hello @support ,

      I've been using cvxpy in the server environment which I installed by running

      !conda install -y -c conda-forge cvxpy
      

      in init.ipynb. But whenever this environment is newly initialized, the module is gone and I have to run this cell again (which takes awfully long).

      Is this normal or is there something wrong with my environment?
      My current workaround is placing these lines before the import

      try:
          import cvxpy as cp
      except ImportError:
          import subprocess
      
          cmd = 'conda install -y -c conda-forge cvxpy'.split()
          rn = subprocess.run(cmd)
      
          import cvxpy as cp
      

      Is there a better way?

      Best regards.

      support 2 Replies Last reply Reply Quote 2
      • support
        support @antinomy last edited by

        @antinomy Sorry for delay, maybe we do not completely understand. When is exactly gone the module? Each strategy has its own environment. So the behavior looks ok, the module should be intalled each time a new strategy is defined. But maybe you mean a different problem, correct?

        A 1 Reply Last reply Reply Quote 0
        • A
          antinomy @support last edited by

          @support
          It's actually the same strategy / environment, not a new one.
          If I haven't used it for a while (say, a few hours or a day) and open it again by clicking on the Jupyter button, it says:
          Initialization of the virtual environment. The notebook will be ready in 15 seconds.
          And when I try to run the strategy that worked fine a few hours or a day ago, I get the ModuleNotFoundError and have to install the module again.
          Everything else is still there as it was before - the strategy, custom files - just not cvxpy.

          support 1 Reply Last reply Reply Quote 0
          • support
            support @antinomy last edited by

            @antinomy Hi, we checked, this is the default behaviour. The solution you used is also the recommended one...

            The reason is that after a timeout, the strategy container is removed and it has to be recreated.

            1 Reply Last reply Reply Quote 0
            • support
              support @antinomy last edited by

              @antinomy Conda may take a lot of time to install just one module. Try to use pip install cvxpy instead.

              A 1 Reply Last reply Reply Quote 0
              • A
                antinomy @support last edited by antinomy

                @support
                Yes, pip is way faster. Thanks!
                I might have found an even faster solution but I guess I have to wait a few hours to find out if it really works.

                Here's what I did:

                1. I created a folder in /root/books called "modules" to install cvxpy there to make it persistent:
                !mkdir modules && pip install --target=modules cvxpy
                
                1. Then if the import fails in the strategy, it creates symbolic links in /usr/local/lib/python3.7/site-packages/ that point to the content of /root/books/modules/
                try:
                    import cvxpy as cp
                except ImportError:
                    import os
                    source = '/root/book/modules/'
                    target = '/usr/local/lib/python3.7/site-packages/'
                    for dirpath, dirnames, filenames in os.walk(source):
                        source_path = dirpath.replace(source, '')
                        target_path = os.path.join(target, source_path)
                        if not os.path.exists(target_path) and not os.path.islink(target_path):
                            os.symlink(dirpath, target_path)
                            continue
                        for file in filenames:
                            source_file = os.path.join(dirpath, file)
                            target_file = os.path.join(target, source_path, file)
                            if not os.path.exists(target_file) and not os.path.islink(target_file):
                                os.symlink(source_file, target_file)
                    import cvxpy as cp
                

                Creating the symlinks only takes 0.07 seconds, so fingers crossed 🙂

                UPDATE (a few hours later):
                It actually worked. When I just reopened the strategy, the environment was newly initialized. First I tried just importing cvxpy and got the ModuleNotFoundError. Then I ran the strategy including the code above: cvxpy was imported correctly and the strategy ran.

                I'm not sure if that solution works for every module because I don't know if pip might also write something to other directories than site-packages.

                Anyway, I'm happy with this solution.
                Regards

                Sjackson3289 1 Reply Last reply Reply Quote 1
                • Sjackson3289
                  Sjackson3289 Banned @antinomy last edited by

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • giant.wallaby
                    giant.wallaby last edited by

                    This post is deleted!
                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Powered by NodeBB | Contributors
                    • Documentation
                    • About
                    • Career
                    • My account
                    • Privacy policy
                    • Terms and Conditions
                    • Cookies policy
                    Home
                    Copyright © 2014 - 2021 Quantiacs LLC.
                    Powered by NodeBB | Contributors