Question: Recompiling a CL system as it is running

Question

Recompiling a CL system as it is running

Answers 2
Added at 2016-12-15 10:12
Tags
Question

I am running a common lisp project that fetches market data every 5 seconds. I made some tweaks to the code and want to update it on the production environment. The event loop is really standard:

(loop
   (fetch-data)
   (sleep 5))

Because of the blocking nature of the loop I don't have the REPL at my disposal.

My question: Can I dynamically update the running code?

I know that I can recompile the project with (asdf:compile-system :system-name)

I also know that I can redefine classes at runtime. (Not that I am using classes in my implementation)

However now that I cannot use the REPL I must somehow load the environment in another REPL. Is there a way to do that? (I am using SBCL)

It seems to me that the cleanest approach would be to just implement asynchronous data fetching.

Answers
nr: #1 dodano: 2016-12-15 15:12

Because of the blocking nature of the loop I don't have the REPL at my disposal.

There are ways around it:

  • start the loop in its own thread.
  • or connect something like SLIME to a running SWANK in your Lisp
  • or your loop could regularly check for code updates

I would also not compile the code inside the program, unless you know that the code actually compiles or you can handle compilation errors. Compiling and loading is also usually slower than just loading.

Note that loading code into a running program is not standardized. With SBCL you could not only have threads, but the threads might run concurrently. There are changes, which might not be thread safe or which are causing problems in a running program. Thus you need to plan what kind of changes you want to do.

One thing which simplifies changing the code: load the update while your loop sleeps, if that is possible within the 5 second window.

Some programs might need some more architecture, like the ability to shut down parts of the program while it is updating these parts.

nr: #2 dodano: 2016-12-15 18:12

Not sure if that's going to work in your situation, but I have used quite successfully the swank::handle-request calling in the loop for REPLing with a running interactive application:

(defun handle-swank-requests ()
  (let ((connection (or swank::*emacs-connection* 
                        (swank::default-connection))))    
    (when connection      
      (swank::handle-requests connection t))))

(defun main-cycle () 
  (loop
     (restart-case
         (progn
           (handle-swank-requests)
           (fetch-data)
           (sleep  5))
       (continue () :report "Continue" (print "Continued after error from SWANK")))))

The restart-case construction allows to not break the loop on errors, and continue it after errors get fixed.

I'm not sure if that approach is bulletproof, AFAIK swank uses threading for requests handling by default, so there may be some synchronization issues, so probably not for production usage, for development / debugging.

AFAIK CEPL uses something similar, may be more sophisticated and robust version. Also nice CEPL demo video.

Source Show
◀ Wstecz