Deadlock between tileLoadQueue and UI thread

Topics: Bugs
Jun 29, 2011 at 1:24 PM

Hello, I'm using GreatMap to display assets on the map. The user can check a vehicle on the left and I'll display an image on the map and then ask the map to Zoom at that location. It works fine ... most of the time. A very important deadlock can happen if a user issues many actions at one (i.e. check 3 assets very fast): the main UI thread is locked and the complete application will freeze.

I've spent the last 2 days to analyse the code and trying to get around this problem. Changing the logic would be too hard for me since I don't know the project enough. BUT, I made it work by doing the following. I hope you can adapt this to make a better fix.

The problem starts when you try to ZOOM while Core.ProcessLoadTask() is calling OnNeedInvalidation().

- Zoom is stuck on the lock(tileLoadQueue) [waiting for tileLoadQueue]

- OnNeedInvalidation() is stuck on this.BeginInvoke() [waiting for the UI ... Zoom]

So, I made it work by using a Monitor.TryEnter in the Zoom (since zoom is not that important):

          if (IsStarted)
          {
            bool IsLocked = false;
            try
            {
              IsLocked = Monitor.TryEnter(this.tileLoadQueue, 0);
              if (IsLocked)
                tileLoadQueue.Clear();
            }
            finally
            {
              if (IsLocked)
                Monitor.Exit(this.tileLoadQueue);
            }
            if (!IsLocked) return;

But then the same thing happen in UpdateBounds(), so:

        IsLocked = Monitor.TryEnter(this.tileLoadQueue, 0);
        if (IsLocked)
        {
          tileDrawingListLock.AcquireWriterLock();
 
[...]                        loadWaitCount = 0;           Monitor.PulseAll(tileLoadQueue);         }       }       finally       {         if (IsLocked)           Monitor.Exit(this.tileLoadQueue);       }       if (!IsLocked) return;

Hope this can help anyone.

Thanks !

 

 

          if (IsStarted)
          {
            bool IsLocked = false;
            try
            {
              IsLocked = Monitor.TryEnter(this.tileLoadQueue, 0);
              if (IsLocked)
                tileLoadQueue.Clear();
            }
            finally
            {
              if (IsLocked)
                Monitor.Exit(this.tileLoadQueue);
            }
            if (!IsLocked) return;
Jun 29, 2011 at 2:51 PM

Hello again, I just upgraded my version to the latest and compared the old and new "Core.cs". GreatMap now use Monitor like I did. I'll try it out but I guess you already came to that conclusion.

Thanks !

Coordinator
Jun 29, 2011 at 2:58 PM

yes, but i'm not sure where exactly it locks, specially with multiply controls using one loadingQueue

Jun 29, 2011 at 8:14 PM

During my test, it always deadlocked while doing Control.Invalidate (at the BeginInvoke line) called from ProcessLoadTask. It seems to crash just while trying to call the BeginInvoke because I tried passing a timeout of 3000ms and it doesn't even timeout ! It just get stuck there.

I think the best solution would be to seperate the tiles loading from the other public services (zoom and display images).

For example, you receive a zoom request, so you just move the map to that location. The same Zoom function then issue a REQUEST to you TileService that he should (on another thread) download missing tile if necessary ... Since both thread use the MainThread for UI, I think you should have "UI" Monitor ... since .NET seems unable to manage it.

What do you think ?

 

Coordinator
Jun 30, 2011 at 7:32 AM

the current version does the same, but how do you know when tile is ready? Yes background thread calls invoke to refresh control, and there locks occur

Coordinator
Jun 30, 2011 at 2:45 PM

..can you make demo app which locks frequently, so i can experiment on it?

Jul 1, 2011 at 12:03 PM

I'll do one monday.

Coordinator
Jul 4, 2011 at 10:51 PM

pushed new version, check the sound! ;}

Jul 22, 2011 at 8:27 AM

Hi Radioman,

I was having this problem too, which I corrected using wizmagisters code.  I've just upgraded to the most recent release and the problem has reappeared, this time in the GMapControl.invalidatorWatch method, specifically this line:

while(!skiped && Core.Refresh.WaitOne() || (Core.Refresh.WaitOne(spanMs, false) || true))

The program freezes on startup at this line, but strangely it can be unstuck by simply right-clicking twice on the programs taskbar icon.  Then it continues and so far hasn't frozen again.

Any ideas?

Coordinator
Jul 22, 2011 at 9:02 AM

hm.. does it stuck always or sometimes, can you share your test project?

Jul 22, 2011 at 9:30 AM

It freezes every time.  The project is too big to upload, I'll see if I can create a smaller test app which does the same thing.

Jul 22, 2011 at 10:23 AM

I haven't tracked the problem down, but I've got a workaround.  The program resizes the main form in the OnShown event, which led to the map freezing.  By hiding the map during the resize, the problem no longer occurrs.

Coordinator
Jul 22, 2011 at 10:30 AM

so you just change form size in OnShown ?

Jul 22, 2011 at 10:40 AM

Yes, the app saves and restores the window position.  By hiding the map while the window position is restored, the deadlock is avoided.  It may be a cross-threading issue elsewhere in the app, as we have a lot of code triggered by the map control and the UserControl it's embedded in.  If I get any further clues I'll post them.

Coordinator
Jul 22, 2011 at 10:43 AM

thanks