In this installment, we'll cover both low-level graphics (using Cairo) and plotting graphs inside GUIs (using Winston). Here again we're relying on infrastructure built by many people, including Jeff Bezanson, Mike Nolta, and Keno Fisher.
The display of the image is handled by Cairo, a C library for two-dimensional drawing. Julia's Cairo wrapper isn't currently documented, so let's walk through a couple of basics first.
If you're new to graphics libraries like Cairo, there are a few concepts that may not be immediately obvious but are introduced in the Cairo tutorial. The key concept is that the Cairo API works like "stamping," where a source gets applied to a destination in a region specified by a path. Here, the destination will be the pixels corresponding to a region of a window on the screen. We'll control the source and the path to achieve the effects we want.
Let's play with this. First, inside a new window we create a Cairo-enabled Canvas for drawing:
using Base.Graphics using Cairo using Tk win = Toplevel("Test", 400, 200) c = Canvas(win) pack(c, expand=true, fill="both")
We've created a window 400 pixels wide and 200 pixels high.
c is our Canvas, a type defined in the
Tk package. Later we'll dig into the internals a bit, but for now suffice it to say that a Canvas is a multi-component object that you can often treat as a black box. The initial call creating the canvas leaves a lot of its fields undefined, because you don't yet know crucial details like the size of the canvas. The call to
pack specifies that this canvas fills the entire window, and simultaneously fills in the missing information in the Canvas object itself.
Note that the window is currently blank, because we haven't drawn anything to it yet, so you can see whatever was lying underneath. In my case it captured a small region of my desktop:
Now let's do some drawing. Cairo doesn't know anything about Tk Canvases, so we have to pull out the part of it that works directly with Cairo:
ctx = getgc(c)
getgc means "get graphics context," returning an object (here
ctx) that holds all relevant information about the current state of drawing to this canvas.
One nice feature of Cairo is that the coordinates are abstracted; ultimately we care about screen pixels, but we can set up user coordinates that have whatever scaling is natural to the problem. We just have to tell Cairo how to convert user coordinates to device (screen) coordinates. We set up a coordinate system using
set_coords, defined in
function set_coords(ctx::GraphicsContext, x, y, w, h, l, r, t, b)
x (horizontal) and
y (vertical) specify the upper-left corner of the drawing region in device coordinates, and
h its width and height, respectively. (Note Cairo uses (0,0) for the top-left corner of the window.)
b are the user coordinates corresponding to the left, right, top, and bottom, respectively, of this region. Note that
set_coords will also
clip any drawing that occurs outside the region defined by
h; however, the coordinate system you've specified extends to infinity, and you can draw all the way to the edge of the canvas by calling
Let's fill the drawing region with a color, so we can see it:
# Set coordinates to go from 0 to 10 within a 300x100 centered region set_coords(ctx, 50, 50, 300, 100, 0, 10, 0, 10) set_source_rgb(ctx, 0, 0, 1) # set color to blue paint(ctx) # paint the entire clip region
Perhaps surprisingly, nothing happened. The reason is that the Tk Canvas implements a technique called double buffering, which means that you do all your drawing to a back (hidden) surface, and then blit the completed result to the front (visible) surface. We can see this in action simply by bringing another window over the top of the window we're using to draw, and then bringing our window back to the top; suddenly you'll see a nice blue rectangle within the window, surrounded by whatever is in the background window(s):
Fortunately, to display your graphics you don't have to rely on users changing the stacking order of windows: call
reveal(c) to update the front surface with the contents of the back surface, followed by
update() (or perhaps better,
update is a fairly generic name) to give Tk a chance to expose the front surface to the OS's window manager.
Now let's draw a red line:
move_to(ctx, -1, 5) line_to(ctx, 7, 6) set_source_rgb(ctx, 1, 0, 0) set_line_width(ctx, 5) stroke(ctx) reveal(c) Tk.update()
We started at a position outside the coordinate region (we'll get to see the clipping in action this way). The next command,
line_to, creates a segment of a path, the way that regions are defined in Cairo. The
stroke command draws a line along the trajectory of the path, after which the path is cleared. (You can use
stroke_preserve if you want to re-use this path for another purpose later.)
Let's illustrate this by adding a solid green rectangle with a magenta border, letting it spill over the edges of the previously-defined coordinate region:
reset_clip(ctx) rectangle(ctx, 7, 5, 4, 4) set_source_rgb(ctx, 0, 1, 0) fill_preserve(ctx) set_source_rgb(ctx, 1, 0, 1) stroke(ctx) reveal(c) Tk.update()
fill differs from
paint in that
fill works inside the currently-defined path, whereas
paint fills the entire clip region.
Here is our masterpiece, where the "background" may differ for you (mine was positioned over the bottom of a wikipedia page):
Images are rendered in Cairo inside a
rectangle (controlling placement of the image) followed by
fill. So far this is just like the simple drawing above. The difference is the source, which now will be a surface instead of an RGB color. If you're drawing from Julia, chances are that you want to display an in-memory array. The main trick is that Cairo requires this array to be a matrix of type
Uint32 encoding the color. The scheme is that the least significant byte is the blue value (ranging from
0xff), the next is green, and the next red. (The most significant byte can encode the alpha value, or transparency, if you specify that transparency is to be used in your image surface.)
Images can generate a buffer of
Uint32 for you. Let's try the one in
using Images img = imread("some_photo.jpg") buf = uint32color(img)' image(ctx, CairoRGBSurface(buf), 0, 0, 10, 10) reveal(c) Tk.update()
Rather than manually calling
fill, we use the convenience method
image(ctx, surf, x, y, w, h) (defined in
h are user-coordinates of your canvas, not pixels on the screen or pixels in your image; being able to express location in user coordinates is the main advantage of using
The image should now be displayed within your window (squashed, because we haven't worried about aspect ratio):
It fills only part of the window because of the coordinate system we've established, where the range
0:10 corresponds to an inset region in the center of the window.
While it's a minor point, note that
CairoRGBSurface takes a transpose for you, to convert from the column-major order of matrices in Julia to the row-major convention of Cairo.
Images avoids taking transposes unless necessary, and is capable of handling images with any storage order. Here we do a transpose in preparation to have it be converted back to its original shape by
CairoRGBSurface. If performance is critical, you can avoid the default behavior of
CairoRGBSurface by calling
CairoImageSurface directly (see the
A basic feature of windows is that they should behave properly under resizing operations. This doesn't come entirely for free, although the grid (and pack) managers of Tk take care of many details for us. However, for Canvases we need to to do a little bit of extra work; to see what I mean, just try resizing the window we created above.
The key is to have a callback that gets activated whenever the canvas changes size, and to have this callback capable of redrawing the window at arbitrary size. Canvases make this easy by having a field,
resize, that you assign the callback to. This function will receive a single argument, the canvas itself, but as always you can provide more information. Taking our image example, we could set
c.resize = c->redraw(c, buf)
and then define
function redraw(c::Canvas, buf) ctx = getgc(c) set_source_rgb(ctx, 1, 0, 0) paint(ctx) set_coords(ctx, 50, 50, Tk.width(c)-100, Tk.height(c)-100, 0, 10, 0, 10) image(ctx, CairoRGBSurface(buf), 0, 0, 10, 10) reveal(c) Tk.update() end
Here you can see that we're aiming to be a bit more polished, and want to avoid seeing bits of the desktop around the borders of our drawing region. So we fill the window with a solid color (but choose a garish red, to make sure we notice it) before displaying the image. We also have to re-create our coordinate system, because that too was destroyed, and in this case we dynamically adjust the coordinates to the size of the canvas. Finally, we redraw the image. Note we didn't have to go through the process of converting to
Uint32-based color again. Obviously, you can use this
redraw function even for the initial rendering of the window, so there's really no extra work in setting up your code this way.
If you grab the window handle and resize it, now you should see something like this:
Voila! We're really getting somewhere now.
Unlike the complete GUI, this implementation doesn't have the option to preserve the image's aspect ratio. However, there's really no magic there; it all comes down to computing sizes and controlling the drawing region and coordinate system.
One important point: resizing the window causes the existing Cairo context(s) to be destroyed, and creates new ones suitable for the new canvas size. One consequence is that your old
ctx variable is now invalid, and trying to use it for drawing will cause a segfault. For this reason, you shouldn't ever store a ctx object on its own; always begin drawing by calling
A Canvas already comes with a set of fields prepared for mouse events. For example, in the complete GUI we have the equivalent of the following:
selectiondonefunc = (c, bb) -> zoombb(imgc, img2, bb) c.mouse.button1press = (c, x, y) -> rubberband_start(c, x, y, selectiondonefunc)
rubberband_start, a function defined in
rubberband.jl, will now be called whenever the user presses the left mouse button.
selectiondonefunc is a callback that we supply; it will be executed when the user releases the mouse button, and it needs to implement whatever it is we want to achieve with the selected region (in this case, a zoom operation). Part of what
rubberband_start does is to bind
selectiondonefunc to the release of the mouse button, via
bb is a
BoundingBox (a type defined in
base/graphics.jl) that will store the region selected by the user, and this gets passed to
selectiondonefunc. (The first two inputs to
img2, store settings that are relevant to this particular GUI but will not be described in detail here.)
mouse inside a
Canvas is an object of type
MouseHandler, which has fields for
release of all 3 mouse buttons and additional ones for motion. However, a few cases (which happen to be relevant to this GUI) are not available in
MouseHandler. Here are some examples of how to configure these actions:
# Bind double-clicks bind(c.c, "<Double-Button-1>", (path,x,y)->zoom_reset(imgc, img2)) # Bind Shift-scroll (using the wheel mouse) bindwheel(c.c, "Shift", (path,delta)->panhorz(imgc,img2,int(delta)))
delta argument for the wheel mouse will encode the direction of scrolling.
Support for the rubber band is provided in the file
navigation.jl, this is a stand-alone set of functions that you should be able to incorporate into other projects. It draws a dashed rectangle employing the same machinery we described at the top of this page, with slight modifications to create the dashes (through the
set_dash function). By now, this should all be fairly straightforward.
However, these functions use one additional trick worth mentioning. Let's finally look at the Tk
type Canvas c::TkWidget front::CairoSurface # surface for window back::CairoSurface # backing store frontcc::CairoContext backcc::CairoContext mouse::MouseHandler redraw function ...
Here we can explicitly see the two buffers, used in double-buffering, and their associated contexts.
c is a
Canvas, simply returns
backcc. This is why all drawing occurs on the back surface. For the rubber band, we choose instead to draw on the front surface, and then (as the size of the rubber band changes) "repair the damage" by copying from the back surface. Since we only have to modify the pixels along the band itself, this is fast. You can see these details in
For many GUIs in Julia, an important component will be the ability to display data graphically. While we could draw graphs directly with Cairo, it would be a lot of work to build from scratch; fortunately, there's an excellent package, Winston, that already does this.
Since there's a nice set of examples of some of the things you can do with Winston, here our focus is very narrow: how do you integrate Winston plots into GUIs built with Tk. Fortunately, this is quite easy. Let's walk through an example:
using Tk using Winston win = Toplevel("Testing", 400, 200) fwin = Frame(win) pack(fwin, expand=true, fill="both")
We chose to fill the entire window with a frame
fwin, so that everything inside this GUI will have a consistent background. All other objects will be placed inside
Next, let's set up the elements, a Canvas on the left and a single button on the right:
c = Canvas(fwin, 300, 200) grid(c, 1, 1, sticky="nsew") fctrls = Frame(fwin) grid(fctrls, 1, 2, sticky="sw", pady=5, padx=5) grid_columnconfigure(fwin, 1, weight=1) grid_rowconfigure(fwin, 1, weight=1) ok = Button(fctrls, "OK") grid(ok, 1, 1)
Finally, let's plot something inside the Canvas:
x = linspace(0.0,10.0,1001) y = sin(x) p = FramedPlot() add(p, Curve(x, y, "color", "red")) Winston.display(c, p) reveal(c) Tk.update()
You'll note that you can resize this window, and the plot grows or shrinks accordingly.
Easy, huh? The only part of this code that is specific to GUIs is the line
Winston.display(c, p), where we specified that we wanted our plot to appear inside a particular Canvas. Of course, there's a lot of magic behind the scenes in Winston, but covering its internals is beyond our scope here.
There's more one could cover, but most of the rest is fairly specific to this particular GUI. A fair amount of code is needed to handle coordinates: selecting specific regions within the 4d image, and rendering to specific regions of the output canvas. If you want to dive into these details, your best bet is to start reading through the
ImageView code, but it's not going to be covered in any more detail here.
Hopefully by this point you have a pretty good sense for how to produce on-screen output with Tk, Cairo, and Winston. It takes a little practice to get comfortable with these tools, but the end result is quite powerful. Happy hacking!