Navigation

    Quantiacs Community

    • Register
    • Login
    • Search
    • Categories
    • News
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    Optimizer not working locally

    Support
    3
    10
    578
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dark.yellowjacket last edited by

      optimizer does not work on my local platform.

      whether i set workers to 1 or to os.cpu_count which is 20.

      1 Reply Last reply Reply Quote 0
      • D
        dark.yellowjacket last edited by

        additionally
        the following complaint

        The environment variable DATA_BASE_URL was not specified. The default value is 'https://data-api.quantiacs.io/'
        NOTICE: The environment variable CACHE_RETENTION was not specified. The default value is '7'
        NOTICE: The environment variable CACHE_DIR was not specified. The default value is 'data-cache'

        1 Reply Last reply Reply Quote 0
        • D
          dark.yellowjacket last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • D
            dark.yellowjacket last edited by dark.yellowjacket

            additional information after trying to optimize THIS IS ON WINDOWS


            KeyboardInterrupt Traceback (most recent call last)
            ~\AppData\Local\Temp\ipykernel_9644\3607707927.py in <module>
            6 strategy_long,
            7 qno.full_range_args_generator(ma_period=range(10, 200, 10)),
            ----> 8 workers = 20 #cpu_count # you can set more workers on your local PC to speed up default =1
            9 )
            10 end = time.time()

            ~\anaconda3\envs\qntdev\lib\site-packages\qnt\optimizer.py in optimize_strategy(data, output_function, argument_generator, stats_function, stats_to_weight, workers)
            38 argument_generator,
            39 stats_to_weight,
            ---> 40 workers
            41 )
            42

            ~\anaconda3\envs\qntdev\lib\site-packages\qnt\optimizer.py in optimize(target_function, argument_generator, result_to_weight, workers)
            61 for w in workers:
            62 w.daemon = True
            ---> 63 w.start()
            64
            65 with progressbar.ProgressBar(max_value=length, poll_interval=1) as bar:

            ~\anaconda3\envs\qntdev\lib\multiprocessing\process.py in start(self)
            110 'daemonic processes are not allowed to have children'
            111 _cleanup()
            --> 112 self._popen = self._Popen(self)
            113 self._sentinel = self._popen.sentinel
            114 # Avoid a refcycle if the target function holds an indirect

            ~\anaconda3\envs\qntdev\lib\multiprocessing\context.py in _Popen(process_obj)
            221 @staticmethod
            222 def _Popen(process_obj):
            --> 223 return _default_context.get_context().Process._Popen(process_obj)
            224
            225 class DefaultContext(BaseContext):

            ~\anaconda3\envs\qntdev\lib\multiprocessing\context.py in _Popen(process_obj)
            320 def _Popen(process_obj):
            321 from .popen_spawn_win32 import Popen
            --> 322 return Popen(process_obj)
            323
            324 class SpawnContext(BaseContext):

            ~\anaconda3\envs\qntdev\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj)
            87 try:
            88 reduction.dump(prep_data, to_child)
            ---> 89 reduction.dump(process_obj, to_child)
            90 finally:
            91 set_spawning_popen(None)

            ~\anaconda3\envs\qntdev\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
            58 def dump(obj, file, protocol=None):
            59 '''Replacement for pickle.dump() using ForkingPickler.'''
            ---> 60 ForkingPickler(file, protocol).dump(obj)
            61
            62 #

            KeyboardInterrupt:

            1 Reply Last reply Reply Quote 0
            • support
              support last edited by

              Hello, sorry for the basic question, did you have some warnings when using the toolbox locally (no optimizer)?

              D 1 Reply Last reply Reply Quote 0
              • D
                dark.yellowjacket @support last edited by

                @support
                No just the complaint indicated on my second comment.

                support 1 Reply Last reply Reply Quote 0
                • support
                  support @dark.yellowjacket last edited by

                  @dark-yellowjacket Ok, thanks. We are aware of optimizer issues. Please see also:

                  https://quantiacs.com/community/topic/249/strategy-optimization-in-local-development-environment-is-not-working

                  https://quantiacs.com/community/topic/233/local-development-error-could-not-convert-string-to-float-nas

                  We are working on that. Sorry for the problems.

                  D 1 Reply Last reply Reply Quote 0
                  • D
                    dark.yellowjacket @support last edited by dark.yellowjacket

                    @support
                    Hello I resolved the issue

                    1. I moved the function and copied pasted the appropriate import s to a ,py file and named it strategy .py

                    2. i commented out the strategy_long(...) function from my notebook

                    3. then :... inserted in the notebook: from strategy import * before running the optimizer.

                    4 ) and it works FYI for asset by asset 20 cores about 2,5 times faster counting wall time

                    NOTE : I think this procedure gets around Windows Python GIL. however, it does not appear to be explained in the python Docs

                    support C 2 Replies Last reply Reply Quote 1
                    • support
                      support @dark.yellowjacket last edited by

                      @dark-yellowjacket Thanks a lot for posting the solution!

                      1 Reply Last reply Reply Quote 0
                      • C
                        captain.nidoran @dark.yellowjacket last edited by

                        Thanks @dark-yellowjacket for post your solution for the optimizer, it works fine for me but could you please go deeper explaining how do you implement the optimized obtained parameter values in the final strategy. I have tried several methods (even in several computers) , obtaining always errors.
                        For example using this template provided by Quantiacs (where i call to "Strategy" I use the same previouslly optimized, and the config.json is the output asset by asset obtained after optimization):

                        
                        import json
                        import xarray as xr
                        import qnt.backtester as qnbk
                        from Strategy import *
                        
                        def optmized_strategy(data, config):
                            results = []
                            for c in config:
                                results.append(strategy_long(data, **c))
                            # align and join results
                            results = xr.align(*results, join='outer')
                            results = [r.fillna(0) for r in results]
                            output = sum(results) / len(results)
                            return output
                        
                        
                        config = json.load(open('config.json', 'r'))
                        
                        # multi-pass
                        # It may look slow, but it is ok. The evaluator will run only one iteration per day.
                        qnbk.backtest(
                            competition_type='stocks_nasdaq100',
                            lookback_period=365,
                            strategy=lambda d: optmized_strategy(d, config),
                         # strategy=strategy_long, # you can check the base strategy too
                            start_date='2006-01-01')
                        

                        It rises the following error:

                        Reloaded modules: Estrategia
                        fetched chunk 1/5 0s
                        fetched chunk 2/5 0s
                        fetched chunk 3/5 0s
                        fetched chunk 4/5 0s
                        fetched chunk 5/5 0s
                        Data loaded 1s
                        Run last pass...
                        Load data...
                        fetched chunk 1/1 0s
                        Data loaded 0s
                        Run strategy...
                        Traceback (most recent call last):
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\spyder_kernels\py3compat.py", line 356, in compat_exec
                            exec(code, globals, locals)
                        
                          File "c:\users\luispc\desktop\quantiacs\q18\prueba.py", line 39, in <module>
                            start_date='2006-01-01')
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\qnt\backtester.py", line 291, in backtest
                            result = strategy_wrap(data, state)
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\qnt\backtester.py", line 270, in <lambda>
                            strategy_wrap = (lambda d, s: strategy(d)) if args_count < 2 else strategy
                        
                          File "c:\users\luispc\desktop\quantiacs\q18\prueba.py", line 37, in <lambda>
                            strategy=lambda d: optmized_strategy(d, config),
                        
                          File "c:\users\luispc\desktop\quantiacs\q18\prueba.py", line 22, in optmized_strategy
                            results.append(strategy_long(data, **c))
                        
                          File "C:\Users\LuisPC\Desktop\quantiacs\Q18\Estrategia.py", line 16, in strategy_long
                            data = data.sel(asset=[asset])
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\xarray\core\dataarray.py", line 1337, in sel
                            **indexers_kwargs,
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\xarray\core\dataset.py", line 2505, in sel
                            self, indexers=indexers, method=method, tolerance=tolerance
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\xarray\core\coordinates.py", line 422, in remap_label_indexers
                            obj, v_indexers, method=method, tolerance=tolerance
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\xarray\core\indexing.py", line 120, in remap_label_indexers
                            idxr, new_idx = index.query(labels, method=method, tolerance=tolerance)
                        
                          File "C:\Users\LuisPC\.conda\envs\qntdev\lib\site-packages\xarray\core\indexes.py", line 242, in query
                            raise KeyError(f"not all values found in index {coord_name!r}")
                        
                        KeyError: "not all values found in index 'asset'"
                        

                        I tried to explain this problem about the implementation in the Backtester in another forum thread time ago, without getting a valid answer, therefore I would appreciate any idea you can give me on this matter.
                        ...Otherwise I am afraid that I will not present algorithms for this contest (Sadly 😢 )
                        Regards.
                        Luis G.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Powered by NodeBB | Contributors
                        • Documentation
                        • About
                        • Career
                        • My account
                        • Privacy policy
                        • Terms and Conditions
                        • Cookies policy
                        Home
                        Copyright © 2014 - 2021 Quantiacs LLC.
                        Powered by NodeBB | Contributors