Discussion:
Some performance notes
Steven Swerling
2006-07-19 15:18:00 UTC
Permalink
Hi,

Today I decided to look into the relation between build settings and
performance as measured by tinyBenchmarks.

First I just tried building a release version of wxsqueak with msvc's
compiler optimized for speed (btw, rob, I believe your Release build
settings do not optimize for speed).

I ran one of my wxsqueak.image on a variety of VMs, and got the
following results:
stock squeak.exe (squeak 3.7)
'218,057,921 bytecodes/sec; 6,830,576 sends/sec'
my release SPSwxsqueakFAST.exe (compiler optimized for speed)
'115,732,368 bytecodes/sec; 5,726,364 sends/sec'
debug wxsqueakd.exe
'54,794,520 bytecodes/sec; 2,581,531 sends/sec'
standard wxsqueak.exe
'53,377,814 bytecodes/sec; 2,507,017 sends/sec'

So just flipping the "optimize" bit gets bytes/sec halfway there, and
sends/sec almost home.

I then replaced the vm sources with the original 3.7 vm sources (from
sourceforge), but still got 115M bytes/sec from the build. So I decided
to build using the "build.bat" file that uses gcc, but w/out the
WXPlugin. This build got me the full performance (218M bytecodes/sec).
Note that this gcc build was on the same sources I with which I had just
built with MSVC -- it just yields a faster executable.

This all suggests that perhaps getting WxPlugin to compile using the
standard make tools might net a little performance increase. Or perhaps
that there is some hidden compiler flags for msvc that will speed things up.

Regards, Steve S
Rob Gayvert
2006-07-19 15:18:00 UTC
Permalink
Post by Steven Swerling
Hi,
Today I decided to look into the relation between build settings and
performance as measured by tinyBenchmarks.
First I just tried building a release version of wxsqueak with msvc's
compiler optimized for speed (btw, rob, I believe your Release build
settings do not optimize for speed).
I ran one of my wxsqueak.image on a variety of VMs, and got the
stock squeak.exe (squeak 3.7)
'218,057,921 bytecodes/sec; 6,830,576 sends/sec'
my release SPSwxsqueakFAST.exe (compiler optimized for speed)
'115,732,368 bytecodes/sec; 5,726,364 sends/sec'
debug wxsqueakd.exe
'54,794,520 bytecodes/sec; 2,581,531 sends/sec'
standard wxsqueak.exe
'53,377,814 bytecodes/sec; 2,507,017 sends/sec'
So just flipping the "optimize" bit gets bytes/sec halfway there, and
sends/sec almost home.
I then replaced the vm sources with the original 3.7 vm sources (from
sourceforge), but still got 115M bytes/sec from the build. So I
decided to build using the "build.bat" file that uses gcc, but w/out
the WXPlugin. This build got me the full performance (218M
bytecodes/sec). Note that this gcc build was on the same sources I
with which I had just built with MSVC -- it just yields a faster
executable.
This all suggests that perhaps getting WxPlugin to compile using the
standard make tools might net a little performance increase. Or
perhaps that there is some hidden compiler flags for msvc that will
speed things up.
Yowser. That's more than a little increase. I get very similar numbers
on my box for all four cases. But this is good news. It's not often you
can get a 2x or 4x speedup with so little effort. For now, I'll at least
change my release version settings to get the 2x increase. Later, I'll
move over to the gcc build.
Steven Swerling
2006-07-19 15:18:00 UTC
Permalink
Post by Rob Gayvert
Yowser. That's more than a little increase. I get very similar numbers
on my box for all four cases. But this is good news. It's not often you
can get a 2x or 4x speedup with so little effort. For now, I'll at least
change my release version settings to get the 2x increase. Later, I'll
move over to the gcc build.
Yeah. The fact that I can build the same sources with gcc and msvc, and
the gcc version is twice as fast (bytecodes/sec), that's encouraging.

In other news, I've been working on an "ImageUtilPlugin" with the goal
of getting Form<->WxBitmap and back. After that, I'm going to try to
implement a WxMorphicPanel. I've poked around a lot in Morphic and have
a pretty clear idea of how to do it on the Morphic side (famous last
words). My guess is that success will hinge on how fast I can turn Forms
into WxBitmaps. I'm cautiously optimistic, at least for Win32, because:
1) SqueakWinIntel.c already has code for using DibSections to update the
display (for wince), and thus I should be able to swipe some of that
stuff for getting a DIB section from a Form; 2) WxWidgets in MSW has a
WxDIB class that is just a windows DIB section (according to a usenet
post by the guy who wrote WxDIB). Or maybe there is a route to setting
the bits directly on the WxBitmap itself (I noticed
WxDIB::ConvertToBitmap includes the line "HBITMAP hbmp =
::CreateDIBitmap", so it looks like a WxBitmap is just a windows DDB --
so I may be able to implement a "Handler" that just calls SetDibBits on
the DDB.). Anyway, that's my current project. Let me know if you have
any thoughts on this.
Cees de Groot
2006-07-19 15:18:00 UTC
Permalink
Post by Steven Swerling
Yeah. The fact that I can build the same sources with gcc and msvc, and
the gcc version is twice as fast (bytecodes/sec), that's encouraging.
Personally, but I'm not an expert, I find it hard to believe that MSVC
produces code that's so crappy that gcc breezes past it by a factor of 2.
So assuming that Rob prefers to live in VisualStudio land, I think it
might be useful maybe to ask around on squeak-dev to see whether anyone
knows what the best MSC options are - there must be something obvious
that's missing...
Steven Swerling
2006-07-19 15:18:00 UTC
Permalink
On Mon, 28 Feb 2005 20:28:43 -0500, Steven Swerling
Post by Steven Swerling
Yeah. The fact that I can build the same sources with gcc and msvc, and
the gcc version is twice as fast (bytecodes/sec), that's encouraging.
Personally, but I'm not an expert, I find it hard to believe that MSVC
produces code that's so crappy that gcc breezes past it by a factor of 2.
I'm not an expert either. In a previous post, I noted that perhaps there
is some compiler flag in msvc that will help. If you don't take that
into account, the statement above makes looks like an anti "M$" rant.
Rather than cast aspersions on msvc per se, the intent is just to show
that much of the performance hit in Wx *might* just come down to how it
is being compiled. For now, the simple fact is that I *can* get a 2x
increase by compiling the same sources, and I haven't found a msvc
compiler incantation that will speed things up.

There is another benefit to getting a gcc build chain working though,
even if it's just for "release" builds. It will make it easier to
compile a unix version of WxSqueak.
Rob Gayvert
2006-07-19 15:18:00 UTC
Permalink
Post by Steven Swerling
In other news, I've been working on an "ImageUtilPlugin" with the goal
of getting Form<->WxBitmap and back. After that, I'm going to try to
implement a WxMorphicPanel. I've poked around a lot in Morphic and have
a pretty clear idea of how to do it on the Morphic side (famous last
words). My guess is that success will hinge on how fast I can turn Forms
1) SqueakWinIntel.c already has code for using DibSections to update the
display (for wince), and thus I should be able to swipe some of that
stuff for getting a DIB section from a Form; 2) WxWidgets in MSW has a
WxDIB class that is just a windows DIB section (according to a usenet
post by the guy who wrote WxDIB). Or maybe there is a route to setting
the bits directly on the WxBitmap itself (I noticed
WxDIB::ConvertToBitmap includes the line "HBITMAP hbmp =
::CreateDIBitmap", so it looks like a WxBitmap is just a windows DDB --
so I may be able to implement a "Handler" that just calls SetDibBits on
the DDB.). Anyway, that's my current project. Let me know if you have
any thoughts on this.
Very cool. WxMorphicPanel has long been on my "things that sound really
cool but I have no idea how hard they are" list.

Judging from the wxWidgets source and the Squeak VM code, it looks like
DDBs are used where possible, so they must be appreciably faster than
DIBs. Do you always need to convert a Form to a wxBitmap, or would it
suffice to draw it in a DC? If the latter, you might be able to do
something along the lines of ioShowDisplay in sqWin32Window.c.

What's your overall scheme? Will morphs draw themselves on a WxCanvas?
And how will events get routed?
Steven Swerling
2006-07-19 15:18:01 UTC
Permalink
Post by Rob Gayvert
Very cool. WxMorphicPanel has long been on my "things that sound really
cool but I have no idea how hard they are" list.
Judging from the wxWidgets source and the Squeak VM code, it looks like
DDBs are used where possible, so they must be appreciably faster than
DIBs. Do you always need to convert a Form to a wxBitmap, or would it
suffice to draw it in a DC? If the latter, you might be able to do
something along the lines of ioShowDisplay in sqWin32Window.c.
What's your overall scheme? Will morphs draw themselves on a WxCanvas?
And how will events get routed?
(sorry long response)

It turns out that form bits and WxBitmap bits are in the same format, so
creating a WxBitmap out of a form is quite fast and easy -- if they are
the same color depth. I haven't pulled off conversion when the Form is
in a non-screen depth yet, and I'm busily stealing code from
ioShowDisplay and friends to try to get this working. I was thinking
last night that I might just defer this for now, since I want to get to
the smalltalk side of the code soon.

Anyway, I can now turn a form into a WxBitmap. I rigged up a little test
where it grabs a Form from the DisplayScreen object and blasts it onto a
WxDC via a WxBitmap. Actually, I do it 100 times in a loop, sending a
different form to the WxDC each time -- it looks to be pretty darned
fast, although I havent formally profiled it. This code creates a brand
new WxBitmap for each draw.

A variety of things I want to get working on the primitive side:

1. Patching part of a form to an existing WxBitmap (instead of creating
a new WxBitmap each time). Required near term, obviously. The rest on
the list may be deferred until after I get the smalltalk/morphic side
working.
2. As you intuited, having a prim call that blasts form bits directly to
a WxDC is likely doable and will be faster. Definitely on the todo list.
3. Deferred until after WxMorphicPanel is working, but on the todo list,
is getting from a WxDC, WxBitmap, or WxImage to a Form.
4. As noted above, I've spent some time on getting forms of alternate
bit depths to display (stealing copious code from ioShowDisplay), but
I'll probably defer this if I don't get it working very soon.
Post by Rob Gayvert
What's your overall scheme? Will morphs draw themselves on a WxCanvas?
And how will events get routed?
My overall scheme on the Morphic side is to:
1. Create a WxMorphicWorldState that updates any WxMorphicCanvases on
the same tick as the dipsplay update. I believe this is easy.
2. Create a WxPasteUpMorph that has an associated WxMorphicCanvas. This
PasteUpMorph will behave in many ways like a World.
3. Create a WxMorphicCanvas (subclassed from WxCanvas, not squeak's
Canvas). Details on reasoning follow. The WxMorphicCanvas will handle
blasting formbits to the WxDC, and also handle adapting Wx
key/mouse/drag-drop events into Morphic events.
4. I may or may have to give WxPasteUpMorphs there own WxHandMorph.
Details below.

[The rest is just a more detailed explanation of the above 4 points]

1. Create a "WxMorphicWorldState" so that I can make a small patch to
the methods update the main squeak World (which is a PasteUpMorph). The
world state will tell it's WxPasteUpMorphs (there could be more than
one) to update their WxDC on the same tick that the main display is
updated. There are already Hooks for Nebraska in WorldState that can be
altered for WxMorhicWorldState to update WxDCs instead of remote canvases.

2. Having a subclass of PasteUpMorph (WxPasteUpMorph), that has it's own
"forceToScreen". Actually, the WxWorldState may come here to update the
Wx display, or may go straight to the WxMorphicCanvas (I haven't decided
yet, and don't think this is a crucial decision). One way or another,
the WxMorphicCanvas winds up blasting the form bits of it's associated
WxPasteUpMorph to its WxDC.

The code that sends the form bits to the screen may or may not use a
WxBitmap -- depends on what I come up with down in the primitives. Since
I've got most of Form->WxBitmap working *now*, I may just go with what
I've got so I can start on the smalltalk side, then optimize this step
later. How the WxMorphicCanvas gets the form canvas bits to the screen
will be invisible to WxPasteUpMorph, so this can be optimized later.

Note that by WxMorphicCanvas, I mean a subclass of WxCanvas, not a
squeak Canvas. I do not believe that having a special squeak canvas is
the way to go. Just let them draw the way they always do, then, in the
final step, instead of blasting the result to the screen, blast it to a
WxDC instead. A side benefit of *not* using a special squeak canvas
would be that most squeak drawing machinery that uses squeak canvases
can be drawn on a WxDC. If it sets form bits, then it can be used by Wx.

I may have to finesse some of the code in WxPasteUpMorph so that it can
act like a "world" in some ways. I'd like it to be able to show flaps,
show a world menu on mouse click, etc.

3. Having WxMorphicCanvas adapt mouse and keyboard events into Morphic
events and dispatch them to the PasteUpMorph. This could be tricky, or
it could be easy, I'm not 100% sure. I've taken a pretty long look at
this, and I'm hoping that I can just feed adapted versions of Wx
key/mouse/drag-drop events straight into the pasteup morph. But I may
have to give each pasteup morph it's own HandMorph and tinker with the
event processing machinery in WxHandMorph. This may be naive -- a proof
of concept version of this is critical enough that I will probably go
staight to this code as soon as the Form->WxDC code is "good enough for
now". But my initial tour of the event code makes me think this is
doable (one way or the other) all on the smalltalk side.
Andreas Raab
2006-07-19 15:18:00 UTC
Permalink
Post by Cees de Groot
Post by Steven Swerling
Yeah. The fact that I can build the same sources with gcc and msvc, and
the gcc version is twice as fast (bytecodes/sec), that's encouraging.
Personally, but I'm not an expert, I find it hard to believe
that MSVC produces code that's so crappy that gcc breezes past
it by a factor of 2.
Actually, it isn't MSVC producing crappy code, but rather GCC producing
outstandingly fast code. But it doesn't do this on its own - a process
called "gnuification" of the VM source code is involved as well (invented by
Ian) which inserts certain optimizations in the VM which are GNU specific
and have no equivalent in MSVC (this is when you get gnu-interp.c instead of
interp.c and use sqGnu.h instead of sq.h - have a look at these files). For
example, we use labeled goto's instead of a case dispatch (this is known to
bring 30-50% in bytecode speed alone since we're cutting out two jumps and a
test per bytecode) and the use of specifically assigned register variables
for the most heavily used local vars in interpret() doesn't hurt either.

So it ain't exactly breezing but rather hard work and (literally) years of
profiling and measuring different options. But the speed difference is
real - MSVC never came even close to ballpark range after I started using
the gnuification process.
Post by Cees de Groot
So assuming that Rob prefers to live in VisualStudio land, I think it
might be useful maybe to ask around on squeak-dev to see whether anyone
knows what the best MSC options are - there must be something obvious
that's missing...
It might be useful to ask around but don't expect anything obvious to "fix"
this problem. Unless MS has added support for those things (which is
possible; I have not been following MSVC due to lack of $$$) you probably
won't get anywhere near the speed of the gnuified VM.

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:00 UTC
Permalink
Post by Andreas Raab
Post by Cees de Groot
Post by Steven Swerling
Yeah. The fact that I can build the same sources with gcc and msvc,
and
Post by Steven Swerling
the gcc version is twice as fast (bytecodes/sec), that's encouraging.
Personally, but I'm not an expert, I find it hard to believe
that MSVC produces code that's so crappy that gcc breezes past
it by a factor of 2.
Actually, it isn't MSVC producing crappy code, but rather GCC
producing outstandingly fast code. But it doesn't do this on its own -
a process called "gnuification" of the VM source code is involved as
well (invented by Ian) which inserts certain optimizations in the VM
which are GNU specific and have no equivalent in MSVC (this is when
you get gnu-interp.c instead of interp.c and use sqGnu.h instead of
sq.h - have a look at these files). For example, we use labeled goto's
instead of a case dispatch (this is known to bring 30-50% in bytecode
speed alone since we're cutting out two jumps and a test per bytecode)
and the use of specifically assigned register variables for the most
heavily used local vars in interpret() doesn't hurt either.
So it ain't exactly breezing but rather hard work and (literally)
years of profiling and measuring different options. But the speed
difference is real - MSVC never came even close to ballpark range
after I started using the gnuification process.
Post by Cees de Groot
So assuming that Rob prefers to live in VisualStudio land, I think it
might be useful maybe to ask around on squeak-dev to see whether anyone
knows what the best MSC options are - there must be something obvious
that's missing...
It might be useful to ask around but don't expect anything obvious to
"fix" this problem. Unless MS has added support for those things
(which is possible; I have not been following MSVC due to lack of $$$)
you probably won't get anywhere near the speed of the gnuified VM.
Cheers,
- Andreas
Andreas - thanks for the detailed explanation! This is amazing stuff. I
suspected the gnuify process was for performance, but I had no idea it
was this dramatic.

I do a fair amount of work with cygwin and gcc as well, so I'm not
partial to any particular tool; I just happened to start with MSVC. But
after seeing these numbers, I'll definitely have to switch over to a gcc
build.
Cees de Groot
2006-07-19 15:18:00 UTC
Permalink
Post by Rob Gayvert
I do a fair amount of work with cygwin and gcc as well, so I'm not
partial to any particular tool; I just happened to start with MSVC. But
after seeing these numbers, I'll definitely have to switch over to a gcc
build.
I didn't want to say this, because it's you doing the hard work, but,
yes... seems like a better idea than trying to get MSC up to speed ;)

Andreas - thanks for the detailed explanation! I was about to set off on a
wild goose chase around MSC command line switches, you saved me quite some
time here.
Andreas Raab
2006-07-19 15:18:00 UTC
Permalink
Rob -
I do a fair amount of work with cygwin and gcc as well, so I'm not partial
to any particular tool; I just happened to start with MSVC. But after
seeing these numbers, I'll definitely have to switch over to a gcc build.
Don't get spoiled. There is no reason why you would have to compile a
*plugin* with GCC (unless you like it). If I remember our last conversation
correctly, then there wasn't any intrinsic need for a VM modification (is
this still correct?) so you might be better off with an external plugin
(compiled with whatever) and the stock VM.

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:00 UTC
Permalink
Post by Andreas Raab
Rob -
Post by Rob Gayvert
I do a fair amount of work with cygwin and gcc as well, so I'm not
partial to any particular tool; I just happened to start with MSVC.
But after seeing these numbers, I'll definitely have to switch over
to a gcc build.
Don't get spoiled. There is no reason why you would have to compile a
*plugin* with GCC (unless you like it). If I remember our last
conversation correctly, then there wasn't any intrinsic need for a VM
modification (is this still correct?) so you might be better off with
an external plugin (compiled with whatever) and the stock VM.
Good point, and I'd certainly like to get it all into an external
plugin. I was able to move some startup and shutdown code into the
plugin as you suggested a few months ago, but there is still one small
change that is necessary for wxSqueak. In sqWin32Window.c, the function
ioProcessEvents calls

while(PeekMessage(&msg,0,0,0,PM_NOREMOVE)) {
GetMessage(&msg,0,0,0);

For wxSqueak this needs to be

while(PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
GetMessage(&msg,stWindow,0,0);

so that the events to non-Squeak windows are not consumed here. Do you
think we could find a way to work in a fix for this in the stock VM?
Cees de Groot
2006-07-19 15:18:00 UTC
Permalink
Post by Rob Gayvert
while(PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
GetMessage(&msg,stWindow,0,0);
It's been ages since I chatted in C with the Win32 API, but this wouldn't
harm regular Squeak at all, would it?
Rob Gayvert
2006-07-19 15:18:00 UTC
Permalink
On Wed, 02 Mar 2005 10:43:03 -0500, Rob Gayvert
Post by Rob Gayvert
while(PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
GetMessage(&msg,stWindow,0,0);
It's been ages since I chatted in C with the Win32 API, but this
wouldn't harm regular Squeak at all, would it?
I think there are some issues here with using Squeak as a browser
plugin, but I'm unclear on how all that works.
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Post by Cees de Groot
Post by Rob Gayvert
while(PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
GetMessage(&msg,stWindow,0,0);
It's been ages since I chatted in C with the Win32 API, but this wouldn't
harm regular Squeak at all, would it?
We just tried it, and no, it wouldn't harm regular Squeak at all. But it
does seem to harm Areithfa Ffenestri support (see
http://minnow.cc.gatech.edu/squeak/3862) quite severely.

Just out of curiosity: Why does wxSqueak have problems with the code that's
in the VM right now? It is the common application mainloop so unless
wxSqueak implements its own mainloop there should be no reason to modify the
above. And I couldn't possibly imagine that you got the VM to work with
anything but the mainloop in interp.c ;-)

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
Post by Cees de Groot
Post by Rob Gayvert
while(PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
GetMessage(&msg,stWindow,0,0);
It's been ages since I chatted in C with the Win32 API, but this
wouldn't harm regular Squeak at all, would it?
We just tried it, and no, it wouldn't harm regular Squeak at all. But
it does seem to harm Areithfa Ffenestri support (see
http://minnow.cc.gatech.edu/squeak/3862) quite severely.
That certainly makes sense -- everything but your main window will be
ignored.
Post by Andreas Raab
Just out of curiosity: Why does wxSqueak have problems with the code
that's in the VM right now? It is the common application mainloop so
unless wxSqueak implements its own mainloop there should be no reason
to modify the above. And I couldn't possibly imagine that you got the
VM to work with anything but the mainloop in interp.c ;-)
Well, this was the fundamental problem in getting two GUI systems to
work together, when each one wants to have the MainLoop. I'm not crazy
about the current scheme, but here's how it works. As we've discussed,
the Squeak loop is modified to ignore events to non-Squeak windows. At
startup, a process is run that checks every few milliseconds for an
event to a wx-window (ignoring events to the Squeak window). When it
gets an event, it passes it through the normal wxWidgets processing. Not
pretty, but it works.

Initially, I had hoped that I could tap into events after they arrived
on the Smalltalk side, assuming they had been augmented with window
information like your're doing with Areithfa Ffenestri. But Squeak
events are handled asychronously, and so much of the wxWidgets event
processing depends on them being handled synchronously that I just
couldn't get it to work.
Cees de Groot
2006-07-19 15:18:01 UTC
Permalink
Post by Rob Gayvert
Initially, I had hoped that I could tap into events after they arrived
on the Smalltalk side, assuming they had been augmented with window
information like your're doing with Areithfa Ffenestri.
Wouldn't a general solution be that Squeak somehow makes available events
that aren't for the main window?

while (event = getEvent())
getHandlerForEvent(event).dispatch(event)

where the Squeak main window would just be one of the handlers? In this
way, every extension/plugin could just hook in and play ball...
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Post by Rob Gayvert
Post by Andreas Raab
We just tried it, and no, it wouldn't harm regular Squeak at all. But it
does seem to harm Areithfa Ffenestri support (see
http://minnow.cc.gatech.edu/squeak/3862) quite severely.
That certainly makes sense -- everything but your main window will be
ignored.
Well, it makes no sense whatsoever to me ;-) If wxWindows runs a mainloop I
would expect it to drain the app event queue and dispatch all the events
(just like the loop in ioProcessEvent does). If it would do so, then clearly
Ffenestri should work (since the events would eventually get into the
WndProc and passed up to Squeak) so the behavior of wxWindows is truly
extraordinary.

It is really important to understand what a Windows app mainloop really
does: If we have, e.g., the standard loop from sqWin32Window.c

while(PeekMessage(&msg,NULL,0,0,PM_NOREMOVE))
{
GetMessage(&msg,NULL,0,0);
DispatchMessage(&msg);
}

then the following happens:
* PeekMessage: Will see if there are any events for *any* window in the
app's event queue without removing it
* GetMessage: Will fetch the next event from the queue which may be for
*any* window
* DispatchMessage: Will deliver the event to the *appropriate* window's
WndProc (which is registered with the window when the window is created)

In other words: A window created by wxWindow *will* receive the event
through the above mechanism in its registered WndProc. A mainloop like the
above run by wxWindow *should* deliver events to *either* the main Squeak
window or any other (e.g., Ffenestri-created) windows. The behavior of
wxWindows apparently not adhering to either of the above two is extremely
strange!

I would expect that the first version works (wxWindows does get the events
synchronously in its WndProc!) *and* that the second version works -
wxWindow should deliver the events to the appropriate WndProc, so you should
be perfectly able to simply not to run ioProcessEvents *at all* (this would
be a good test - if Squeak doesn't work when wxWindows runs the mainloop
something is horribly broken).
Post by Rob Gayvert
Well, this was the fundamental problem in getting two GUI systems to work
together, when each one wants to have the MainLoop. I'm not crazy about
the current scheme, but here's how it works. As we've discussed, the
Squeak loop is modified to ignore events to non-Squeak windows. At
startup, a process is run that checks every few milliseconds for an event
to a wx-window (ignoring events to the Squeak window). When it gets an
event, it passes it through the normal wxWidgets processing. Not pretty,
but it works.
How does it check for the events? (can I see the source code?)
Post by Rob Gayvert
Initially, I had hoped that I could tap into events after they arrived on
the Smalltalk side, assuming they had been augmented with window
information like your're doing with Areithfa Ffenestri. But Squeak events
are handled asychronously, and so much of the wxWidgets event processing
depends on them being handled synchronously that I just couldn't get it to
work.
I think you have a major bug in your thought process - as far as I can tell
(and as far as I know Windows) your windows should get the events just fine
(see above explanation). Unless there is some very strange magic in the main
loop of wxWindows (do you have the source for the mainloop?) this ought to
work with no further modifications.

Cheers,
- Andreas
Cees de Groot
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
DispatchMessage(&msg);
Ah. so this gets done already (sorry, my last windoze hacking experience
was I think when Win 3.1 was hot).
Post by Andreas Raab
I think you have a major bug in your thought process - as far as I can
tell (and as far as I know Windows) your windows should get the events
just fine (see above explanation). Unless there is some very strange
magic in the main loop of wxWindows (do you have the source for the
mainloop?) this ought to work with no further modifications.
If it ought to work I vote for making it work :). Whether it's a problem
with the thought process or with wxWidgets (having browsed the wxWindows
source code, I'm in favor of blaming everything on them, I am quite
unforgiving to people who still use preprocessor hacks well over a decade
after C++ got templates), we should fix this.

Personally, I think I'd rather see a patch in wxWidgets than in Squeak.
It'd be soo much cleaner if wxSqueak would build as just a plugin...
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
Post by Rob Gayvert
Post by Andreas Raab
We just tried it, and no, it wouldn't harm regular Squeak at all.
But it does seem to harm Areithfa Ffenestri support (see
http://minnow.cc.gatech.edu/squeak/3862) quite severely.
That certainly makes sense -- everything but your main window will be
ignored.
Well, it makes no sense whatsoever to me ;-) If wxWindows runs a
mainloop I would expect it to drain the app event queue and dispatch
all the events (just like the loop in ioProcessEvent does). If it
would do so, then clearly Ffenestri should work (since the events
would eventually get into the WndProc and passed up to Squeak) so the
behavior of wxWindows is truly extraordinary.
Why doesn't this make sense? If you use
PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
you're only getting messages for the main Squeak window. And in my wx
check, I'm only peeking for messages to windows other than the main
Squeak window.
Post by Andreas Raab
It is really important to understand what a Windows app mainloop
really does: If we have, e.g., the standard loop from sqWin32Window.c
while(PeekMessage(&msg,NULL,0,0,PM_NOREMOVE))
{
GetMessage(&msg,NULL,0,0);
DispatchMessage(&msg);
}
* PeekMessage: Will see if there are any events for *any* window in
the app's event queue without removing it
* GetMessage: Will fetch the next event from the queue which may be
for *any* window
* DispatchMessage: Will deliver the event to the *appropriate*
window's WndProc (which is registered with the window when the window
is created)
In other words: A window created by wxWindow *will* receive the event
through the above mechanism in its registered WndProc. A mainloop like
the above run by wxWindow *should* deliver events to *either* the main
Squeak window or any other (e.g., Ffenestri-created) windows. The
behavior of wxWindows apparently not adhering to either of the above
two is extremely strange!
Well, I'm certainly no Win32 expert, so I can't say whether wxWidgets is
strange, but its MainLoop does more than Peek/Get/Dispatch. If you have
the stomach for it, take a look at the attachment, evtloop.cpp.

If I revert the original ioProcessEvents and turn off my event process,
events do get delivered to wx windows, but it behaves very badly and
generally crashes in short order. And if events destined for wx windows
are passed on to wxApp::ProcessMessage from ioProcessEvents, it works a
bit better, but still crashes. The only scheme I've found that works
consistently is to poll for wx events in a separate Squeak process. I
don't like it, but it works.
Post by Andreas Raab
I would expect that the first version works (wxWindows does get the
events synchronously in its WndProc!) *and* that the second version
works - wxWindow should deliver the events to the appropriate WndProc,
so you should be perfectly able to simply not to run ioProcessEvents
*at all* (this would be a good test - if Squeak doesn't work when
wxWindows runs the mainloop something is horribly broken).
That would be a good test, but I wouldn't be surprised if the funky
stuff that goes on in ProcessMessage will screw it up for Squeak.
Post by Andreas Raab
Post by Rob Gayvert
Well, this was the fundamental problem in getting two GUI systems to
work together, when each one wants to have the MainLoop. I'm not
crazy about the current scheme, but here's how it works. As we've
discussed, the Squeak loop is modified to ignore events to non-Squeak
windows. At startup, a process is run that checks every few
milliseconds for an event to a wx-window (ignoring events to the
Squeak window). When it gets an event, it passes it through the
normal wxWidgets processing. Not pretty, but it works.
How does it check for the events? (can I see the source code?)
You can get all the wxSqueak source from my website, and all of the
wxWidgets source from wxwidgets.org.
The key part of the check is this:

if (PeekMessage(&msg,0,0,0,PM_NOREMOVE)) {
if (msg.hwnd != stWindow){
GetMessage(&msg,0,0,0);
if (!((wxSqueakApp*)wxTheApp)->ProcessMessage(&msg)){
TranslateMessage(&msg);
DispatchMessage(&msg);
}

If it weren't for the ProcessMessage() call, this would be a "normal" loop.
Post by Andreas Raab
Post by Rob Gayvert
Initially, I had hoped that I could tap into events after they
arrived on the Smalltalk side, assuming they had been augmented with
window information like your're doing with Areithfa Ffenestri. But
Squeak events are handled asychronously, and so much of the wxWidgets
event processing depends on them being handled synchronously that I
just couldn't get it to work.
I think you have a major bug in your thought process - as far as I can
tell (and as far as I know Windows) your windows should get the events
just fine (see above explanation). Unless there is some very strange
magic in the main loop of wxWindows (do you have the source for the
mainloop?) this ought to work with no further modifications.
I hope you're right. It would be much cleaner to handle all events in a
unified way, and eliminate the horribly inefficient event process in Squeak.




-------------- next part --------------
///////////////////////////////////////////////////////////////////////////////
// Name: msw/evtloop.cpp
// Purpose: implements wxEventLoop for MSW
// Author: Vadim Zeitlin
// Modified by:
// Created: 01.06.01
// RCS-ID: $Id: evtloop.cpp,v 1.25 2004/07/30 22:54:25 VZ Exp $
// Copyright: (c) 2001 Vadim Zeitlin <***@dptmaths.ens-cachan.fr>
// License: wxWindows licence
///////////////////////////////////////////////////////////////////////////////

// ============================================================================
// declarations
// ============================================================================

// ----------------------------------------------------------------------------
// headers
// ----------------------------------------------------------------------------

#if defined(__GNUG__) && !defined(NO_GCC_PRAGMA)
#pragma implementation "evtloop.h"
#endif

// For compilers that support precompilation, includes "wx.h".
#include "wx/wxprec.h"

#ifdef __BORLANDC__
#pragma hdrstop
#endif

#ifndef WX_PRECOMP
#include "wx/window.h"
#include "wx/app.h"
#endif //WX_PRECOMP

#include "wx/evtloop.h"

#include "wx/tooltip.h"
#include "wx/except.h"
#include "wx/ptr_scpd.h"

#include "wx/msw/private.h"

#if wxUSE_THREADS
#include "wx/thread.h"

// define the array of MSG strutures
WX_DECLARE_OBJARRAY(MSG, wxMsgArray);

#include "wx/arrimpl.cpp"

WX_DEFINE_OBJARRAY(wxMsgArray);
#endif // wxUSE_THREADS

// ----------------------------------------------------------------------------
// helper class
// ----------------------------------------------------------------------------

// this object sets the wxEventLoop given to the ctor as the currently active
// one and unsets it in its dtor
class wxEventLoopActivator
{
public:
wxEventLoopActivator(wxEventLoop **pActive,
wxEventLoop *evtLoop)
{
m_pActive = pActive;
m_evtLoopOld = *pActive;
*pActive = evtLoop;
}

~wxEventLoopActivator()
{
// restore the previously active event loop
*m_pActive = m_evtLoopOld;
}

private:
wxEventLoop *m_evtLoopOld;
wxEventLoop **m_pActive;
};

// ============================================================================
// wxEventLoop implementation
// ============================================================================

wxEventLoop *wxEventLoopBase::ms_activeLoop = NULL;

// ----------------------------------------------------------------------------
// ctor/dtor
// ----------------------------------------------------------------------------

wxEventLoop::wxEventLoop()
{
m_shouldExit = false;
m_exitcode = 0;
}

// ----------------------------------------------------------------------------
// wxEventLoop message processing
// ----------------------------------------------------------------------------

void wxEventLoop::ProcessMessage(WXMSG *msg)
{
// give us the chance to preprocess the message first
if ( !PreProcessMessage(msg) )
{
// if it wasn't done, dispatch it to the corresponding window
::TranslateMessage(msg);
::DispatchMessage(msg);
}
}

bool wxEventLoop::PreProcessMessage(WXMSG *msg)
{
HWND hwnd = msg->hwnd;
wxWindow *wndThis = wxGetWindowFromHWND((WXHWND)hwnd);

// this may happen if the event occured in a standard modeless dialog (the
// only example of which I know of is the find/replace dialog) - then call
// IsDialogMessage() to make TAB navigation in it work
if ( !wndThis )
{
// we need to find the dialog containing this control as
// IsDialogMessage() just eats all the messages (i.e. returns true for
// them) if we call it for the control itself
while ( hwnd && ::GetWindowLong(hwnd, GWL_STYLE) & WS_CHILD )
{
hwnd = ::GetParent(hwnd);
}

return hwnd && ::IsDialogMessage(hwnd, msg) != 0;
}

#if wxUSE_TOOLTIPS
// we must relay WM_MOUSEMOVE events to the tooltip ctrl if we want it to
// popup the tooltip bubbles
if ( msg->message == WM_MOUSEMOVE )
{
wxToolTip *tt = wndThis->GetToolTip();
if ( tt )
{
tt->RelayEvent((WXMSG *)msg);
}
}
#endif // wxUSE_TOOLTIPS

// allow the window to prevent certain messages from being
// translated/processed (this is currently used by wxTextCtrl to always
// grab Ctrl-C/V/X, even if they are also accelerators in some parent)
if ( !wndThis->MSWShouldPreProcessMessage((WXMSG *)msg) )
{
return false;
}

// try translations first: the accelerators override everything
wxWindow *wnd;

for ( wnd = wndThis; wnd; wnd = wnd->GetParent() )
{
if ( wnd->MSWTranslateMessage((WXMSG *)msg))
return true;

// stop at first top level window, i.e. don't try to process the key
// strokes originating in a dialog using the accelerators of the parent
// frame - this doesn't make much sense
if ( wnd->IsTopLevel() )
break;
}

// now try the other hooks (kbd navigation is handled here): we start from
// wndThis->GetParent() because wndThis->MSWProcessMessage() was already
// called above
for ( wnd = wndThis->GetParent(); wnd; wnd = wnd->GetParent() )
{
if ( wnd->MSWProcessMessage((WXMSG *)msg) )
return true;
}

// no special preprocessing for this message, dispatch it normally
return false;
}

// ----------------------------------------------------------------------------
// wxEventLoop running and exiting
// ----------------------------------------------------------------------------

bool wxEventLoop::IsRunning() const
{
return ms_activeLoop == this;
}

int wxEventLoop::Run()
{
// event loops are not recursive, you need to create another loop!
wxCHECK_MSG( !IsRunning(), -1, _T("can't reenter a message loop") );

// ProcessIdle() and Dispatch() below may throw so the code here should
// be exception-safe, hence we must use local objects for all actions we
// should undo
wxEventLoopActivator activate(&ms_activeLoop, this);

// we must ensure that OnExit() is called even if an exception is thrown
// from inside Dispatch() but we must call it from Exit() in normal
// situations because it is supposed to be called synchronously,
// wxModalEventLoop depends on this (so we can't just use ON_BLOCK_EXIT or
// something similar here)
#if wxUSE_EXCEPTIONS
for ( ;; )
{
try
{
#endif // wxUSE_EXCEPTIONS

// this is the event loop itself
for ( ;; )
{
#if wxUSE_THREADS
wxMutexGuiLeaveOrEnter();
#endif // wxUSE_THREADS

// generate and process idle events for as long as we don't
// have anything else to do
while ( !Pending() && (wxTheApp && wxTheApp->ProcessIdle()) )
;

// if the "should exit" flag is set, the loop should terminate
// but not before processing any remaining messages so while
// Pending() returns true, do process them
if ( m_shouldExit )
{
while ( Pending() )
Dispatch();

break;
}

// a message came or no more idle processing to do, sit in
// Dispatch() waiting for the next message
if ( !Dispatch() )
{
// we got WM_QUIT
break;
}
}

#if wxUSE_EXCEPTIONS
// exit the outer loop as well
break;
}
catch ( ... )
{
try
{
if ( !wxTheApp || !wxTheApp->OnExceptionInMainLoop() )
{
OnExit();
break;
}
//else: continue running the event loop
}
catch ( ... )
{
// OnException() throwed, possibly rethrowing the same
// exception again: very good, but we still need OnExit() to
// be called
OnExit();
throw;
}
}
}
#endif // wxUSE_EXCEPTIONS

return m_exitcode;
}

void wxEventLoop::Exit(int rc)
{
wxCHECK_RET( IsRunning(), _T("can't call Exit() if not running") );

m_exitcode = rc;
m_shouldExit = true;

OnExit();

// all we have to do to exit from the loop is to (maybe) wake it up so that
// it can notice that Exit() had been called
//
// in particular, we do *not* use PostQuitMessage() here because we're not
// sure that WM_QUIT is going to be processed by the correct event loop: it
// is possible that another one is started before this one has a chance to
// process WM_QUIT
::PostMessage(NULL, WM_NULL, 0, 0);
}

// ----------------------------------------------------------------------------
// wxEventLoop message processing dispatching
// ----------------------------------------------------------------------------

bool wxEventLoop::Pending() const
{
MSG msg;
return ::PeekMessage(&msg, 0, 0, 0, PM_NOREMOVE) != 0;
}

bool wxEventLoop::Dispatch()
{
wxCHECK_MSG( IsRunning(), false, _T("can't call Dispatch() if not running") );

MSG msg;
BOOL rc = ::GetMessage(&msg, (HWND) NULL, 0, 0);

if ( rc == 0 )
{
// got WM_QUIT
return false;
}

if ( rc == -1 )
{
// should never happen, but let's test for it nevertheless
wxLogLastError(wxT("GetMessage"));

// still break from the loop
return false;
}

#if wxUSE_THREADS
wxASSERT_MSG( wxThread::IsMain(),
wxT("only the main thread can process Windows messages") );

static bool s_hadGuiLock = true;
static wxMsgArray s_aSavedMessages;

// if a secondary thread owning the mutex is doing GUI calls, save all
// messages for later processing - we can't process them right now because
// it will lead to recursive library calls (and we're not reentrant)
if ( !wxGuiOwnedByMainThread() )
{
s_hadGuiLock = false;

// leave out WM_COMMAND messages: too dangerous, sometimes
// the message will be processed twice
if ( !wxIsWaitingForThread() || msg.message != WM_COMMAND )
{
s_aSavedMessages.Add(msg);
}

return true;
}
else
{
// have we just regained the GUI lock? if so, post all of the saved
// messages
//
// FIXME of course, it's not _exactly_ the same as processing the
// messages normally - expect some things to break...
if ( !s_hadGuiLock )
{
s_hadGuiLock = true;

size_t count = s_aSavedMessages.Count();
for ( size_t n = 0; n < count; n++ )
{
MSG& msg = s_aSavedMessages[n];
ProcessMessage(&msg);
}

s_aSavedMessages.Empty();
}
}
#endif // wxUSE_THREADS

ProcessMessage(&msg);

return true;
}
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Hi -
Post by Rob Gayvert
Why doesn't this make sense? If you use
PeekMessage(&msg,stWindow,0,0,PM_NOREMOVE))
you're only getting messages for the main Squeak window. And in my wx
check, I'm only peeking for messages to windows other than the main
Squeak window.
Yes, that part makes sense. The other didn't (why it wouldn't work just the
way it is).
Post by Rob Gayvert
Well, I'm certainly no Win32 expert, so I can't say whether wxWidgets is
strange, but its MainLoop does more than Peek/Get/Dispatch. If you have
the stomach for it, take a look at the attachment, evtloop.cpp.
I see. My initial guess would be that there might be a problem with

wxEventLoop *wxEventLoopBase::ms_activeLoop = NULL;

not being set correctly.
Post by Rob Gayvert
If I revert the original ioProcessEvents and turn off my event process,
events do get delivered to wx windows, but it behaves very badly and
generally crashes in short order.
Any chance of finding out where and why? The code doesn't look too ugly and
while there may be a few things which don't work it doesn't look like it
should just crash on you. Unless, of course, there is a bug (?) in something
like:

wxWindow *wndThis = wxGetWindowFromHWND((WXHWND)hwnd);

If code like this isn't aware that there might indeed be hwnds which are not
owned by wx, things might go nasty.
Post by Rob Gayvert
And if events destined for wx windows
are passed on to wxApp::ProcessMessage from ioProcessEvents, it works a
bit better, but still crashes.
Again, any chance of finding out where it crashes?
Post by Rob Gayvert
The only scheme I've found that works
consistently is to poll for wx events in a separate Squeak process.
I don't like it, but it works.
Post by Andreas Raab
I would expect that the first version works (wxWindows does get the
events synchronously in its WndProc!) *and* that the second version
works - wxWindow should deliver the events to the appropriate WndProc,
so you should be perfectly able to simply not to run ioProcessEvents
*at all* (this would be a good test - if Squeak doesn't work when
wxWindows runs the mainloop something is horribly broken).
That would be a good test, but I wouldn't be surprised if the funky
stuff that goes on in ProcessMessage will screw it up for Squeak.
It doesn't look too funky to me. And again, if it crashes, knowing where it
crashes would be tremendously helpful.
Post by Rob Gayvert
Post by Andreas Raab
Post by Rob Gayvert
At startup, a process is run that checks every few
milliseconds for an event to a wx-window (ignoring events to the
Squeak window). When it gets an event, it passes it through the
normal wxWidgets processing. Not pretty, but it works.
How does it check for the events? (can I see the source code?)
You can get all the wxSqueak source from my website, and all of the
wxWidgets source from wxwidgets.org.
if (PeekMessage(&msg,0,0,0,PM_NOREMOVE)) {
if (msg.hwnd != stWindow){
GetMessage(&msg,0,0,0);
if (!((wxSqueakApp*)wxTheApp)->ProcessMessage(&msg)){
TranslateMessage(&msg);
DispatchMessage(&msg);
}
if (msg.hwnd != stWindow){
Does this crash? (this would clearly point to a problem in the wx libs).
Post by Rob Gayvert
If it weren't for the ProcessMessage() call, this would be a "normal" loop.
Yes, and it should work just fine with Squeak.

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
Post by Rob Gayvert
And if events destined for wx windows
are passed on to wxApp::ProcessMessage from ioProcessEvents, it works a
bit better, but still crashes.
Again, any chance of finding out where it crashes?
Okay, here's where it gets interesting. If the wx pre-processing is
applied, simple isolated events work fine. But if an event comes in
during a callback, it crashes in a completely random way. What's
happening here is a sequence like:
interpret -> ioProcessEvents -> wxProcessMessage -> [...] ->
interpret() -> ioProcessEvents -> ...

With a separate process polling for wx events, re-entry of the
interpreter occurs only during a primitive call.
Post by Andreas Raab
Post by Rob Gayvert
if (PeekMessage(&msg,0,0,0,PM_NOREMOVE)) {
if (msg.hwnd != stWindow){
GetMessage(&msg,0,0,0);
if (!((wxSqueakApp*)wxTheApp)->ProcessMessage(&msg)){
TranslateMessage(&msg);
DispatchMessage(&msg);
}
if (msg.hwnd != stWindow){
Does this crash? (this would clearly point to a problem in the wx libs).
No, it works fine, so the wx pre-processing doesn't seem to hurt.
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Post by Rob Gayvert
Post by Andreas Raab
Again, any chance of finding out where it crashes?
Okay, here's where it gets interesting. If the wx pre-processing is
applied, simple isolated events work fine. But if an event comes in during
a callback, it crashes in a completely random way. What's happening here
interpret -> ioProcessEvents -> wxProcessMessage -> [...] ->
interpret() -> ioProcessEvents -> ...
With a separate process polling for wx events, re-entry of the interpreter
occurs only during a primitive call.
An RSTTT (Really Simple Thing To Try) would be to do, say,:

ioProcessEvents {
static int inCallBack = 0;

if(!inCallBack) {
/* only do wxProcessMessage if not in callback */
inCallBack := 1;
wxProcessMessage(...);
}

}

Does this work? (if it does work reliably, we just solved the problem)
Post by Rob Gayvert
Post by Andreas Raab
Post by Rob Gayvert
if (msg.hwnd != stWindow){
Does this crash? (this would clearly point to a problem in the wx libs).
No, it works fine, so the wx pre-processing doesn't seem to hurt.
Good, good. Very good. Now on for the next try: What happens if you stub out
the whole ioProcessEvents() loop (yes, everything) and just run the loop
from the other process? Everything still fine?

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
Post by Rob Gayvert
Post by Andreas Raab
Again, any chance of finding out where it crashes?
Okay, here's where it gets interesting. If the wx pre-processing is
applied, simple isolated events work fine. But if an event comes in
during a callback, it crashes in a completely random way. What's
interpret -> ioProcessEvents -> wxProcessMessage -> [...] ->
interpret() -> ioProcessEvents -> ...
With a separate process polling for wx events, re-entry of the
interpreter occurs only during a primitive call.
ioProcessEvents {
static int inCallBack = 0;
if(!inCallBack) {
/* only do wxProcessMessage if not in callback */
inCallBack := 1;
wxProcessMessage(...);
}
}
Does this work? (if it does work reliably, we just solved the problem)
This doesn't crash, but it doesn't do much either. This prevents any wx
events from getting processed once you're in a callback, so when this
happens the wx window stops responding.
Post by Andreas Raab
Post by Rob Gayvert
Post by Andreas Raab
Post by Rob Gayvert
if (msg.hwnd != stWindow){
Does this crash? (this would clearly point to a problem in the wx libs).
No, it works fine, so the wx pre-processing doesn't seem to hurt.
Good, good. Very good. Now on for the next try: What happens if you
stub out the whole ioProcessEvents() loop (yes, everything) and just
run the loop from the other process? Everything still fine?
Not quite -- now nothing happens in the Squeak window. But that's
because the Squeak event recording needs access to the "lastMessage",
which is set at the start of ioProcessEvents. If I add a hook to set
lastMessage from my wx event process, everything seems fine.

But where does this leave us? It shows that we can move the event loop
out of ioProcessEvents, but this is a big change to the stock VM, and
we're back to an inefficient polling process. What do you have in mind next?
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Rob,
Post by Rob Gayvert
This doesn't crash, but it doesn't do much either. This prevents any wx
events from getting processed once you're in a callback, so when this
happens the wx window stops responding.
Interesting.
Post by Rob Gayvert
Post by Andreas Raab
Good, good. Very good. Now on for the next try: What happens if you stub
out the whole ioProcessEvents() loop (yes, everything) and just run the
loop from the other process? Everything still fine?
Not quite -- now nothing happens in the Squeak window. But that's because
the Squeak event recording needs access to the "lastMessage", which is set
at the start of ioProcessEvents. If I add a hook to set lastMessage from
my wx event process, everything seems fine.
But where does this leave us? It shows that we can move the event loop out
of ioProcessEvents, but this is a big change to the stock VM, and we're
back to an inefficient polling process. What do you have in mind next?
Well, there are two possibilites as far as I can tell at this point: Either,
there is something in the wx mainLoop() which really wants to be run and
isn't invoked by any of the current variants. We should be able to test this
by calling interpret() from *within* the wx main loop. Try changing the main
Squeak window proc (in sqWin32Window.c) to call interpret() ... uh say ...
on WM_CREATE or so. Then, call the wx mainloop instead of interpret() from
the VM and see if that works. If the problem is really in the "outer"
mainloop this should probably work.

Or, and that's what I'm fearing right now, wx has a problem with reentrant
windows messages. The only way to know for sure is probably to look around
if anyone else has seen such a problem (and mind you it could just be an
issue with one specific message - that's all it takes for a recursive
crash). One thing that would indeed be useful is to find out after which
recursive message the system crashes (if it were a specific one or two we
might be able to handle these on the plugin level).

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Either, there is something in the wx mainLoop() which really wants to
be run and isn't invoked by any of the current variants. We should be
able to test this by calling interpret() from *within* the wx main
loop. Try changing the main Squeak window proc (in sqWin32Window.c) to
call interpret() ... uh say ... on WM_CREATE or so. Then, call the wx
mainloop instead of interpret() from the VM and see if that works. If
the problem is really in the "outer" mainloop this should probably work.
Or, and that's what I'm fearing right now, wx has a problem with
reentrant windows messages. The only way to know for sure is probably
to look around if anyone else has seen such a problem (and mind you it
could just be an issue with one specific message - that's all it takes
for a recursive crash). One thing that would indeed be useful is to
find out after which recursive message the system crashes (if it were
a specific one or two we might be able to handle these on the plugin
level).
I just tried out a test that demonstrates that the real culprit is the
callback mechanism. I stripped out all of the wx initialization, event
processes, and wx event processing from my image and vm, so it should be
possible to reproduce this with a stock image. Here's the setup.

In Squeak, a callback process is started as follows:

semaphore := Semaphore new.
Wx setCallbackSemaphore: (Smalltalk registerExternalObject:
semaphore). "sets semaIndex"
[ [ true ] whileTrue: [
semaphore wait.
Transcript show: 'processing callback'; cr.
Wx returnFromCallback. "longjmp(jmpbuf)"
]] fork.

In the VM, a callback is initiated with:

interpreterProxy->signalSemaphoreWithIndex(semaIndex);
if (setjmp(jmpbuf) == 0) {
interpret();
}

The results:

1. If the callback is initiated from Squeak by a primitive call, it can
be called any number of times without a problem.

2. If the callback is initiated periodically (say once per second) from
ioProcessEvents, it dies very predictably after 15 iterations.

So either (a) I'm doing something wrong with this semaphore; or (b) this
semaphore can't be safely signaled from within the
checkForInterrupts/ioProcessEvents context. Or maybe it needs to be
signaled differently in this situation?
Andreas Raab
2006-07-19 15:18:01 UTC
Permalink
Post by Rob Gayvert
I just tried out a test that demonstrates that the real culprit is the
callback mechanism. I stripped out all of the wx initialization, event
processes, and wx event processing from my image and vm, so it should be
possible to reproduce this with a stock image. Here's the setup.
The first question coming to my mind is: Does wx use any kind of
exception handling? IIRC, then wx is C++ right?! If so, exception
handling might play a significantly different role here.
Post by Rob Gayvert
semaphore := Semaphore new.
semaphore). "sets semaIndex"
[ [ true ] whileTrue: [
semaphore wait.
Transcript show: 'processing callback'; cr.
Wx returnFromCallback. "longjmp(jmpbuf)"
]] fork.
That's what I would expect.
Post by Rob Gayvert
interpreterProxy->signalSemaphoreWithIndex(semaIndex);
if (setjmp(jmpbuf) == 0) {
interpret();
}
That too. Except that (with multiple callbacks) one would expect that
jmpbuf is dynamically allocated and passed to the supporting call to "Wx
returnFromCallback" somehow...
Post by Rob Gayvert
1. If the callback is initiated from Squeak by a primitive call, it can
be called any number of times without a problem.
2. If the callback is initiated periodically (say once per second) from
ioProcessEvents, it dies very predictably after 15 iterations.
If correct, this is indeed interesting (and fixable if need be). But
what exactly qualifies as "being initiated by a primitive call"? E.g.,
in almost all situation ioProcessEvents is called when a primitive is
called (the rare exception are true interrupt checks for long-taking
primitives). Or do you have "special" primitives in mind?
Post by Rob Gayvert
So either (a) I'm doing something wrong with this semaphore; or (b) this
semaphore can't be safely signaled from within the
checkForInterrupts/ioProcessEvents context. Or maybe it needs to be
signaled differently in this situation?
There is only one way to signal the semaphore and that's the above one
(assuming you're running single-threaded; for multi-threaded execution
signalSemaWithIndex() isn't always thread-safe and that's why win32 uses
asyncSignalSemaWithIndex() in some places but this shouldn't be an issue
here).

So I would say it is extremely unlikely that it's related to how/if the
semaphore gets signaled. I think it's something with the callbacks.

Cheers,
- Andreas
Rob Gayvert
2006-07-19 15:18:01 UTC
Permalink
Post by Andreas Raab
Post by Rob Gayvert
I just tried out a test that demonstrates that the real culprit is
the callback mechanism. I stripped out all of the wx initialization,
event processes, and wx event processing from my image and vm, so it
should be possible to reproduce this with a stock image. Here's the
setup.
The first question coming to my mind is: Does wx use any kind of
exception handling? IIRC, then wx is C++ right?! If so, exception
handling might play a significantly different role here.
Yes, wx is definitely C++. But it doesn't use exception handling itself:

http://www.wxwidgets.org/manuals/2.5.3/wx_exceptionsoverview.html
Post by Andreas Raab
Post by Rob Gayvert
interpreterProxy->signalSemaphoreWithIndex(semaIndex);
if (setjmp(jmpbuf) == 0) {
interpret();
}
That too. Except that (with multiple callbacks) one would expect that
jmpbuf is dynamically allocated and passed to the supporting call to
"Wx returnFromCallback" somehow...
Yes, this example is slightly simplified. But I have only one handler
for all wx events, so the jmpbufs are statically allocated, with one for
each level of recursion on interpret().
Post by Andreas Raab
Post by Rob Gayvert
1. If the callback is initiated from Squeak by a primitive call, it
can be called any number of times without a problem.
2. If the callback is initiated periodically (say once per second)
from ioProcessEvents, it dies very predictably after 15 iterations.
If correct, this is indeed interesting (and fixable if need be). But
what exactly qualifies as "being initiated by a primitive call"? E.g.,
in almost all situation ioProcessEvents is called when a primitive is
called (the rare exception are true interrupt checks for long-taking
primitives). Or do you have "special" primitives in mind?
I should have said "external primitive"; that is, a call to a simple
function in my WxPlugin that signals the semaphore and calls
interpret(). I added a trace to see how ioProcessEvents was being
entered, and you're correct -- in this test, most of the time it comes
from the primitive ioRelinquishProcessorForMicroseconds, with a few
scattered occurrences of ioGetNextEvent and the commonSend section of
the interpreter loop.

But I misstated one important detail in this test: the results are valid
only if the callback loop in Squeak is forked at highestPriority. At
normal priority, (1) crashes also.

And here's another interesting tidbit. If I lengthen the callback
interval in (2), it will go on much longer. But the instant I try to
bring up a process browser, it crashes.

So this suggests something is amiss with processes and the re-entrant
call to interpret(). Some time ago I tried to exert more control over
what happens here by suspending the current process before calling
interpret(), and then resuming this process after the
returnFromCallback. This improved matters somewhat, but still
occasionally crashed.

I think all of this is also related to a secondary problem: if I try to
return a value directly from a wx primitive that indirectly kicks off a
callback (e.g., WxFrame>>createStatusBar causes an #onSize event before
it returns), the return value is usually wrong. I've worked around this
by having all of the wx primitives set their return values in static
location, which are then retrieved with a second primitive call, but
this shouldn't be necessary.

My guess is that someone's stack may get messed up if the callback
process is not the first process when interpret() is re-entered. Does
that sound plausible?

Steven Swerling
2006-07-19 15:18:00 UTC
Permalink
Post by Andreas Raab
So it ain't exactly breezing but rather hard work and (literally) years
of profiling and measuring different options. But the speed difference
is real - MSVC never came even close to ballpark range after I started
using the gnuification process.
That explains a lot. Thanks.

What is your preferred ide for gcc?
Andreas Raab
2006-07-19 15:18:00 UTC
Permalink
Post by Steven Swerling
What is your preferred ide for gcc?
Emacs.

Cheers,
- Andreas
Loading...