This is a different excerpt from the same sequence as previously posted. This time the audio corresponding to each pair is included.
I’m going to experiment with the filtering process so that there are more lower intensity (range of flow in a pair) pairs included in the training set in order to increase smoothness. Right now there are a lot of very intense high-movement pairs. I’ve also considered initiating the network with different weight intensities to encourage an ebb and flow. This would depend on how the over-trained network (still processing) turns out.
I created a new training set using three of the Transformers films directed by Michael Bay. This choice was made due to aspect ratio considerations. Some of these films include cuts between scenes with radically different aspect ratios (formats). In some cases, where there were few tall scenes, I just cropped them to 2.4:1 to match the rest of the film. The transformers films selected include Transformers (2007), Revenge of the Fallen (2009) and Dark of the Moon (2011). The total number of frames from all three is a little more than 600,000 frames. After filtering for flows with a larger range than 2, 62,637 pairs remain (about 1.5 hours of duration at 23.976 fps).
The smoothness of the original sequence (including the discontinuities between subsequent films), is 44.1042. I trained a model where every flow vector is presented to the network five times, which took about 14 hours. The following image shows the weights of the model for the first few minutes. At this stage we’re looking at raw weights, not the closest corresponding training samples (flows between pairs of frames), and no redundant frames are removed.
The smoothness here (measured as the average difference between subsequent pairs) is 7.98023, the most smooth sequence we’ve seen to date. This number will go up when we look at the training samples associated with each time-slot, rather than raw weights. The following image shows the same period where the weights are replaced by their nearest training samples.
The smoothness of the actual training samples arranged by SOM is much lower, at 33.2548. As you can see, there are a lot of redundant frames where neighboring time-slots are associated with the same frame. There were 25,226 pairs that were kept from a total of 62,637. This means only 40% of the series is preserved after the removal of redundant pairs, shrinking a ~1.5 hour duration down to 35mins. The following image is the version where redundant frames are removed.
The results are quite good and seem pretty smooth perceptually; I’d like like to get more of a cinematic length so perhaps I’ll try the sort method on the whole set and also over the weekend over-train a model over more iterations.
I realized after publishing that last post that I could resolve the issue with repetition issue with the SOM results simply by removed redundant pairs. The results shift the SOM method past the sorting method with more flow / cohesion as well as just as much intensity. Slightly to my surprise, the version with redundant pairs removed is even more intense. There are also sections of fast panning that feel cohesive, as if the movement is not repeated, but continuous (as in predicted), jump to around 40s in the video below. I think there are two things happening here, first very fast pans are much more abstract, so we focus on direction more than the movement of recognizable objects. I also realized the repetition of similar flow vectors reminds me of looking at the landscape go by at high speed; your eyes jump and track, and jump and track; interleaving recognition with ambiguous movement. That’s a bit like the experience here. The following video is an excerpt of the same SOM sequence previously posted, except with the redundant pairs of frames removed.
I also trained a (compressed) model over-night with around 800,000 training iterations and a length half that of the previous model. The resulting composition is less smooth that the previous results, with an inter-pair average different of 30.0156 (compared to 14.6374 for the previous model). This number should be higher due to both the compression (same information in a smaller length), and over-training (each model weight being closer to the training samples). I also realized that the SOMs to date, with their repetition of pairs, skew the measure of flow: The repeated pairs mean some inter-pair differences are 0, which pushes down the average significantly. Considering the above method of removing the repetitions, I don’t think the compression significantly improves slow.
The compressed model still has lots of repetition of pairs, see following image; this shows the original flows associated with each model weight (position in time), not the weights themselves. The aspect ratio is different because it’s half the length (half the number of weights in the model).
The compressed and over-trained version does not seem significantly better in terms of either reducing the repeated pairs, nor the flow or aesthetic of movement. Despite the number of parameters (1920, maybe the most I’ve even used in a SOM), the model seems to be doing a good job arranging pairs by similarity without a lot of work tuning hyper-parameters (e.g. length, number of iterations, etc.). I took a look at the video with redundant pairs and really don’t like them pairs; this video is the version with those repeated pairs removed. Note that interesting zoom early in the sequence.
So the next steps are to jump into prepping source material and then I can start tinkering with all that data.
Following from the previous post, I’ve done a little analysis on the first model and also made a video excerpt. As I mentioned, the previous image (shown again below) is the raw weights, but since we’re using this to organize pairs of frames, what matters more is the arrangement frame flows according to the model.
Modelled Flow
The following image is generated by code that compares every flow (movement in a pair of frames) to each of the model weights. I’m using the model to order the original samples. Disregard the contrast; it’s normalized for visual readability. The actual flow values are quite small so without contrast they’re nearly all grey. Note the similar structure (rhythm over time) to the model above.
SOM Arranged Flows
The first thing to note is that there tends to be more than one flow (original pair) associated with the same model weight (column representing the diversity of flow). This leads to the same pair of frames being placed in subsequent slots in the video sequence. The degree of smoothness of the SOM arrangement is 14.6374 (average difference in flow in subsequent frames), compared to:
9.02628 for the raw weights in the SOM model
11.3053 for the original sequence
18.7964 for the smoothest of the sorted sequences
The repetition of the same pair of frames in sequence in the video really stands out as a flickering that can be pretty intense. In sections where there is less of this repetition, the results are looking pretty interesting; see the following video excerpt:
The repetition of pairs of frames is not surprising since the length of the sequence matches the number of input frames. This is generally not great for ML methods (the number of parameters matching the number of input patterns), but I have used it before to preserve length (in this case) or size (in the case of “Machines of the Present Consume the Imaginations of the Past“). Over-training has solved this where the model becomes nearly identical to the training data (not what you usually want), but in this case still arranged by similarity (what we do want). Today I trained a model with half the number of pairs (half the length), and that compression should reduce this repetition of frames, which I’ll take a look at and discuss soon. All in all, I’m pretty happy with these results and the smoothness is working OK. I think with some tweaking I’ll like these results as much as the sorting method. All of this just points to wanting more data and letting the flow dictate the length. After that, I’ll get to the drudgery of prepping the other sources and start on
After some reflection of the results sorting the whole sequence, as previously posted, I realized that part of what makes the end part of the composition work is the intensity of movement; i.e. there is a large range of values in the flow (amount of movement). I made a few explorations filtering the flows keeping only those pairs with a range of flow above a certain threshold. I also changed the reference Pair from the first few frames to the pair of frames that have the largest range of flow. These results are quite a bit better without the slow start; the movement feels more consistent over the ~30min (yes, I’m filtering out about 2 hours of pairs based on their movement being more subtle). I did not bother putting an excerpt of that up on vimeo because it looks about the same as the end of the whole sorted sequence. The insight being that not that much of a Bay film is actually Bayhem. It would be pretty exhausting of a whole film was only the action sequences, but that’s exactly what I’m going for in this project!
Based on these reflections, I decided to maximize (hopefully) both flow and diversity by using multiple films from the Transformers franchise directed by Bay. With the filtering by range of flow and more diversity of movements from different films, the results will hopefully be both more contiguous in flow (smooth) and more discontinuous in subject (different frames, films, settings, characters, etc.). It’ll take some time to organize those materials; the plan is to concatenate all the source and treat it like one giant super film where the gaps between the films would be read just like another scene change; this would keep me from making a lot more changes to the current codebase.
I also got to training a SOM on the filtered flow and while I’ve not made a video yet, the flows themselves are looking promising. The following is the flow over time of the latest sorted version, with pair 62597 as the reference; the smoothness (average of inter-pair flow difference) is 57.369.
Sorted Flow
The corresponding results from the SOM, using the same filtered flows as above, has a very high-smoothness with an average of inter-pair flow difference of 9.02628, even smoother than the original film, see following. This is a bit misleading through, as this is the smoothness of the model, not the arrangement of frames using that model. I’ll work on that next. Note the intensity of flows is more evenly distributed, whereas the middle of the sorted version above has a longer period in the middle of less flow.
Following the previous post, I ran the optical flow and sorting methods on the entire film, using the first frames as the reference pair. Due the number of pairs, images of the feature vectors are too large for an image to be very readable (103343 x 1920px), so I won’t bother posting that. As I’m interested in flow smoothness over the rearranged sequence, I will do an analysis of the distance between subsequent sorted vectors to get a sense of how many discontinuities there are; i.e. how many large distances between subsequent pairs there are. This will also be useful to compare to the ML method of organizing pairs. I can also use the number of pairs to normalize the sum (average) of inter-pair differences so that the whole sequence can be comparable to the previous 1-20,000 frame experiment (which I re-ran with the first frames as reference). I can use that to determine if indeed having more frames increases the smoothness of the composition over time, as previously hypothesized.
Due to the way that optical flow is represented (one channel for x flow, another for y flow for all pixels arranged in a grid) it’s hard to make sense of whether it’s working and the features have meaning. In order to validate the features before jumping into the machine learning part, I wanted to take a look at the raw flow data and see if it was intelligible, which it is (more or less). The following image shows each feature vector for a pair of frames as a single column. The flow arrays are reshaped and the x and y displacements interleaved (i.e. x,y,x,y). So there’s a interlace-y effect. That being said, we can see that in the first few frames, there is a dominant movement in one direction (darker pixels), a bit of a lull and then more complex movement. This does match the first scene of the film.
So the rows have little meaning with all that inter-leaving of x and y movements in the rows and columns of the original frames. The next step is to choose a reference pair (2981) with clear movement (in this case the central to lower right of frame moving from leftwards and the central left moving rightwards), as shown here:
Here is the corresponding optical flow vector, stretched out across columns to be readable:
Still pretty hard to visually decide this data is meaningful, so I’ll sort all the features according to their distance to this reference pair. If the closest pair also shows a similar pattern of movement, then we can confirm the features are meaningful in terms of representing movement. I can then also make a test sequence where movements close to the reference movement are placed together close in time; the result should be a sequence that starts with right to left movement in the central-lower right of the frame and then descend into something different as each pair’s distance (similarity) to the reference increase over the sequence.
One of the new works for the New Media Gallery residency I proposed was something that’s been in the back of my mind for years. Michael Bay’s transformers movies were my first experience of this idea that cuts in cinema are increasing and scenes getting shorter and shorter. Combine that with Bay’s tendency for camera movement and CGI, and the result is sections of Bay’s movies I find unintelligible. Bayhem (Bay + mayhem) is has been coined to describe Bay’s intense style, and the title of this work. The idea of the project is an analysis of movement in Bay’s highest grossing film, “Transformers: Dark of the Moon” (2011), according to movement. Looking over the film briefly, I’m not sure there is a single shot in which the camera does not move.
The first step my my analysis is tracking scene changes. This is because those pairs for frames will not be intelligible in terms of movement analysis. Using an OpenCV example as a basis, and using the Hellinger distance between histograms of hue and saturation, I’m looking at those distances to detect scene changes and omit those from the motion analysis. The following plot shows the first ~20,000 frames. The dashed horizontal line shows the scene detection threshold, the line plot the Hellinger distance, and each detected scene change is labelled by it’s frame number.
I looked over all these pairs and they generally look pretty good, out of 100 pairs of frames, 84 were confirmed scene changes and 16 were very bright flashes (of which there are many in the film). These are usually one to a few frames where the contrast and colour changes significantly. As those would also cause issues with the movement analysis, I’ve decided to keep them as “scene changes”. Some actual scene changes have less contrast than these flashes; consider the scene change between frame 3473 & 3474, where the distance (0.479038) is very close to the detection threshold:
Compare this to the 10593 & 10594 pair of frames which has a much higher (0.96969) histogram distance (difference):
It’s better for the detector to be quite aggressive (better to include a flash as a cut than miss an actual cut) because the project is an emphasis on flow. The motion analysis will be used to produce a new moving image work.
Now that the MoveDayA final prints are complete and installed, I can finalize the ConversationVanessa prints that correspond to the two portrait displays at the gallery. The following images show the corresponding wave, histograms and accumulation.
Following from this post, I dialed in the aesthetic for stripes. In general, things are more constrained, fewer possible, and lower, frequencies and fewer possible layer offsets with tweaks to the weights; these are also a little more saturated (by decreasing the probability of very transparent layers). The results are more visually interesting and feel more connected with the painting histories that inspired the project. They’re actually a bit of a call-back to the first sketches for this project; that set of images was actually generated for my first Canada Council grant application for the Zombie project, which was submitted 10 years ago!
After a significant number of test prints I finally picked up the final large scale prints yesterday, and then were installed that afternoon! The final prints are shown on the top row below (hist on left, waves on right). The bottom row shows the test prints made to date leading up to the final. There is still a smaller photographic print we’re waiting for that may only arrive tomorrow (the day of the opening). After the opening I’ll go back to print work and finish off the Vanessa side of things and then this installation will finally be complete! Some new test prints will go up through that process.
I was feeling iffy about the fine line version, so ended up doing high frequency (fine-line) and medium frequency sets. After seeing them on the Zombie in the gallery, I decided not to go in the fine-line direction. The idea of using fine-lines was to make something that contrasts more with the gradients and also more of an AI anti-aesthetic (dense regular patterns are still a significant weakness in image generators). I did not find they looked interesting and moved too far away from the dialogue with painting. I’ve included both sets below for reference, and also an a couple poor images of these explorations on the Zombies photographed at the gallery.
High-Frequency (open image to see up close)Medium DensityZombies with FramesZombies without Frames
I shared a little time with both in the gallery and the medium (or even lower) density needs quite a bit of work, which I will focus on today.
I ended up getting a quite rough unordered dithering shader working, which does improve the banding, but also changes the aesthetic a fair bit due to the change of contrast. The shader itself is kluged together from multiple places online and could use some work. The pseudo random number generator is a bit weird (and appears in many shader examples using noise), but I did not have time to give it much investigation in terms of distribution or even range. My thought is just to add a random value between -1 and +1 (multiplied by a small constant) to all colour channels. That is not how it works ATM, and seems to add or subtract depending on the colour channels. The following gallery shows the dithered and non-dithered versions so you can see the differences in contrast (there are also jpeg artifacts):
DitheringNo dithering
The spreading of the values is not a problem (light blue area at the top being larger on the left from the dithering), as I would have liked to see even lower frequency waves than I ended up with to avoid hard-edge clipping. The following gallery shows some variation of the dithered compositions:
With this, I have something working for the opening in a couple days! At least for one of the Zombies. I’m going to work on getting the Dense Hard-Edge aesthetic for the second one. After the opening I’ll get back to Face Detection and Instagram uploading.
With the opening coming up soon, I decided to focus on some aesthetic explorations that could be running in the gallery while I get the Instagram and face detection stuff going. The following image is a montage of 100 soft-edge compositions. I ended up tweaking a fair bit to make sure there are no hard edges; the frequency is locked to one (trough to trough) cycle over the whole screen, so it can be offset without introducing a hard edge. I’ve also included a screen-resolution detail of one composition:
100 Random Soft-Edge CompositionsCherry-Picked Composition from Above Set.
I am quite happy for these in general, but due to the blending and some quite low-contrast gradients, I’m not happy with the banding (obvious above). I’m looking at using a dithering shader to use noise to break up these bands.
In prep for the soft opening, I’ve not been blogging much. This is more of a photo dump of the current state of work in process at the gallery; we’ll see how much changes in a week! This includes some of rough unedited photos I took as a teaser.
Last week I moved the Zombies to the gallery and they’ve been hung! The following images show a bit of the unpacking process. It’ll be a little while before they’re up and running, but they have power and network so I can start on that work. This is the first time the “Zombie Formalist“s, “Machines of the Present Consume the Imaginations of the Past” (canvases and two moving images), and kitsch will be shown together as they were envisioned!
I have not posted much recently as I’ve been doing a little too much debugging and configuration work, but things at the gallery are really shaping up. Following are some WiP images of the now titled “A Shaping in Time” (aka Quantum Studio) installation. Significant progress has been made since these images were taken, come visit the gallery to see! The ConversationVanessa needs a lot of work, but MoveDayA is looking great.
Final Print Images
The following images are actually going to the printer after quite a few test prints. As recently as yesterday I made code changes realizing there were bugs in my code that became apparent with the increase of margins from 1 to 2 inch; I did not take into account the way that my sections of the plot (to get around the cairo raster resolution limit) could happen in between time-steps; i.e. I specify the sections of the plot in time, with each point in the wave located at 1/30 time sections) which lead to some sections trying to plot data that did not exist (i.e. they were not locked to time-steps), so that took a little more math to round down to the previous time-step in plotting so that the sections matched the data rather than arbitrary pixel boundaries (width/5 sections).
When I first wrote the above paragraph I thought I was done (with MoveDayA), but again the printer experienced crashes due to these very large files and I have to tweak things yet again. I’ve dropped the resolution from 1200 to 600 dpi and generated a full-size image that was confirmed to not crash the system by the printer. So the images in the rest of this post may change slightly due to potentially needing to tune the line thickness for the lower resolution.
The following gallery shows photos of the multi-alpha print test previously posted. These images were taken outside in variable cloud and include only colour (no contrast) correction. The fine lines in isolation are looking much better at the upper end of the opacity scale, e.g. 0.32 and 0.35 are both looking good. The peaks (top row) are looking a little too dark at 0.35. I think splitting the difference is the best approach, so we’re looking at an alpha of 0.335, compared to 0.2 in the initial test print.
I’ve sent off the file below to the printer for a final test print for the waves plot. I’ve also included the previous test print for comparison.
In addition to the accumulations previously posted, and included below, I realized I missed a couple features for sorting (f-number and focus distance). Some of this accumulation code was written when I was in Paris for the Quantum Studio; one of the developments was sorting layers according to EXIF data from the time-lapse. Only one of these new accumulations was strong enough to post. I’ve included all four below, with the two strongest in the bottom row.
There are some interesting edge effects in the bottom right image (the new one), due to sorting by EXIF focus distance. Once I get the larger plot prints in good shape I can select the final image for this accumulation before moving onto the finalization of ConversationVanessa.
I got the test print back from the printer this week and while fixing the DPI certainly controls that variable, it appears the range of brightness for these fine lines is quite a bit different. My lightest lines appear much darker on-screen than they do in the print. The following gallery shows some poor photos of the test print for reference (with no contrast adjustment) so you can see how close the colour of the faint lines are compared to the paper.
Line thickness is looking pretty good, it’s really just the darkness that needs work. As I previously posted, the challenge is getting the fine lines dark enough without the most dense parts completely blackening out. So we’re back to arrays of subsections of the plot with different transparency values. The following image was sent off to the printer yesterday.
My concern with this test print is that the peaks (on the right of the image above) end up blacking out before the light lines get enough weight. The alternative is to non-linearly adjust the contrast (value curves), which is only possible because these images are now raster. The difficulty is the adjustment would be quite arbitrary as the initial test print has proven a significant gap between on-screen and on-print contrast. I’ll take it one step at a time and see how the next test print looks.
After my last post, I realized I had messed up frame indices (there is one more frame than there are inter-frame differences in these sequences); the consequence was that the last frame for one of the sequences (ConversationNaomi) was selected by kmeans, but that frame does not exist. Due to error I reran the kmeans frame selection for ConversationVanessa with the appropriate frame indices. I did not think it was worth repeating this for the other short-listed sequences. When I ran that code over the four short-listed sequences, I also generated only 300 waves per iteration, rather than the 1000 waves per iteration & frame I used for MoveDayA. These two errors effect previously posted waves. On the plus side, the change has resulted in wave structures that make the ConversationVanessa print more interesting. The following images show the ConversationVanessa (top) and MoveDayA (bottom) test prints for reference, see the previous post for comparison.
ConversationVanessaMoveDayA
Interesting that the second cluster of higher frequency waves that we see in MoveDayA also now appears in ConversationVanessa. Part of the reason there is quite a bit of change in these waves due to parameter changes is normalization. When I say “300” vs “1000” waves, I mean for each frame that many pure sine-waves are generated and summed. For plots these aggregate waves for each frame are normalized; when I talk about a plot having “6400” waves, I mean 8 random iterations for each of the 800 layers/frames for the moving image work. In the moving image work, waves accumulate every 5 mins. In each case summing and normalizing may happen at different intervals: The results all have similar character, but also differ in that normalization happens sometimes before and sometimes after one or more wave accumulation(s). It also gives you a bit of a sense of complexity; each plot actually shows the behaviour of 6,400,000 pure sinewaves: 8 iterations of 1000 pure sinewaves for each of the 800 layers/frames. The following images show the full time-series for Conversation Vanessa (top), the corresponding plot for MoveDayA, and a selection of details for Conversation Vanessa.
ConversationVanessaMoveDayADetails of Conversation Vanessa Waves
These may still change based on the results of the test print for MoveDayA. I just got the email that my test print is ready! More details on that soon.
Since my first attempt at looking at the wave structures to determine which sequence to prioritize for the New Media Gallery residency I tweaked the wave-generator (the upper frequency limit in particular) and thus the previous work is outdated. I went back, this time actually doing the kmeans selection of 800 frames for each of the short-listed sequences. The following plot shows the middle-left section of all four, including MoveDayA for reference; this time more comparable due to the same number of frames and using the same wave-generation parameters.
Top to Bottom: Conversation Naomi, Conversation Vanessa, Les Recollets and Move Day A.
These have not changes much, but ConversationNaomi and MoveDayA diverged quite a bit and the former is quite a nice wave! ConversationVanessa is still very strong and LesRecollets remains as the least interesting (most peaky). I suspect the dominant frequency is due to the yellow cast of the over-exposure and direct sunlight because the sun came out mid sequence. I’d be tempted to colour correct that sequence to remove that yellow cast if I’m to make a work specifically for this sequence; that’s outside the scope of the New Media Gallery show. The following images are details of Conversation Naomi and Conversation Vanessa at full resolution:
Conversation NaomiConversation Vanessa
These results don’t push me hard enough to abandon my plans to focus on Conversation Vanessa next. Once I get the test print for MoveDayA, I’ll be able to get moving pretty quick on the prints for MoveDayA and I’m going to get started on the moving image component of Conversation Vanessa next week!
Now that I’ve decided to open up MoveDayA to changes, that means regenerating the “accumulations” for the 800 frames selected by kmeans used in the other components of the installation (moving images, waves, histograms). The following are my top three, where the largest image is my favourite.
This has not been finalized as I’ll wait to see how the test print for the waves looks to see if any more tweaks are needed, but I’ve written new code to generate nice histograms corresponding to the 800 frames (for MoveDayA) I’m now working with. I’ve also included the wave plot below for reference, since they should look a bit balanced; this is not that easy as there are many more waves than histograms (6400 waves vs 800 histograms). I’ve kept the line thickness the same as for the waves, but increased the opacity value to 0.4 (from 0.2). See the following full plot alongside waves plus a set of details.
I realized after tweaking the wave generation parameters (i.e. the upper frequency limit), this will change how the waves look for all sequences; I’ll re-generate waves for all the short-listed sequences to see how they now look, in particular for the Vanessa sequence.
After implementing the changes to the plot renderer to generate raster directly, as previously discussed, I realized things looks really different; outputs are very faint even compared to what I’ve been seeing with the PDF output rasterized in gimp. There’s a significant difference in the aesthetic when rendering directly to raster. Not only do lines look lighter with the same opacity settings, but also the anti-aliasing seems much more poor. After more testing, the anti-aliasing issue is due to those ultra-fine lines, where the thickness is only about 1px, at 1200dpi, so there is not much to work with. Increasing the line thickness to 0.04 (from 0.01), smooths out the curves quite a bit (due to effective anti-aliasing) with only a marginal increase of line size (from ~1 to ~2px thickness). The balance of peak and other areas of the plot are looking pretty good, following is a low-res raster and full resolution details of the revised test print (which I sent to the printer yesterday).
I also attached a low-res version of the full size 144,000 x 20,904 pixel raster (constructed from 5 separate plots to get around the cairo resolution limit):
I’ll report back when I see the test print and will shift my work over to finishing the other two prints for MoveDayA: the histogram plot and the accumulation corresponding to the 800 frames / layers used in the installation and wave prints.
While in the gallery I wanted to take a crack at pushing the moving image so I tried a 1/600 upper frequency limit. It looked slow, but was nice contrast with the intensity of the flashing of the other channel and other work in the space. As I was in the gallery, I did not get a chance to take a look at the wave plots, which I only got to last night. While the slower movement is good, and the longer peak time means I can increase the alpha (from 0.21 to 0.3). We’ve lost that second cluster of waves at higher frequency, so I think I’m going back to 1/400 and I’ll share a little more time with the work at 1/400 today. Today’s my last day in the gallery this week so I won’t be able to reflect on the moving image in relation to wave plots before my self-imposed deadline for the test print. For reference I’ve attached a few of the 1/600 plots here as well as the previously posted test print.
1/600
1/400 Test Print
While at the gallery I got my first look at the moving image with 800 waves. I was initially worried I was going to have to change my contrast adjustment equation, which was quite a lot of effort. After some tweaking (including generating more initial waves before the first frame is rendered) making it more “peaky”. At 800 frames the moving image work has a lot more detail so the peak is less pronounced with more stacked layers. The work also evolves much more slowly than it did with 250 frames and stays quite interesting up into the accumulation of even 1000 sine-waves. While the details are not worked out for installation, I’m leaning to moving from option A (a work that evolves over the course of one day with 250 frames) to option B (something that could change slowly over the course of a week and stay interesting even with many waves having accumulated). I’m currently running a test at the gallery so the next time I see it it will have been over 6 days. If that’s too peaky (which manifests in more dark and sparse imagery with a super high contrast burst in the middle) I can use that as a starting point; to determine a max number of waves I’m writing the number of accumulations to disk and periodic screenshots to get a sense of density over the week I’m not present.
I changed the upper frequency limit to a lower frequency (from 1/100 to 1/200 and 1/400), to draw out the length of the peak. I also took a look at these (for 250 frames / layers) on my machine at home. The results are looking good! They certainly decrease the density around the peak, which should make printing less hard to balance out. As I’m now producing raster directly for the printer, this leaves less variables to control for; I may not need as many print sketches as I had anticipated. The following images show the full plot (at the proper aspect ratio); I had previously failed to take into account left and right margins, so previous plots were a bit squished along the x axis. Note how the change of upper frequency limit has caused a secondary cluster at a dominant frequency (which I had previously seen) that makes the wave-form even more interesting:
I think I have a solution to the raster problem. After more debugging, I realized I was unable to generate raster at this size directly from ggplot2 either, and managed to narrow down the problem to a cairo (the vector graphics library used in both ggplot2 and in GIMP to generate raster from vectors) limitation of a max image dimension of 32,767px. So it seems there is no easy way to create a single image this big from a PDF nor render directly. I did a quick test and I managed to create a seamless composite by breaking the width of the image into sections in ggplot2 — each less than 32,767px wide. I extended this to a full-scale approach with 5 sections and it appears pixel-perfect! While going through all this back-and-forth about the printing process I’ve been reflecting on opening MoveDayA to changes; as I previously mentioned, running the existing code on displays at the NMG was certainly inspiring in that direction and there are some changes I would like to make. Here is a little teaser of what happened in the gallery this past week:
The images in the foreground are the two channels for the quantum project, and the projection behind it is a proxy for a new work titled “Bayhem”. It was nice to see them in the same space and it reminded me of some aesthetic continuity in my work.
Regarding tuning the works to the NMG machines: that will entail experimenting with more frames because those machines have 12GB VRAM vs the 3GB of my own GPU. I was thinking that would fit nicely with 1000 frames (making the 6000 waves benchmark for printing make sense). A little calculation shows that will be a little too much for the 12GB GPU (which is actually 11ish); 800 is a more reasonable choice (though I won’t be able to test this until Tuesday at the earliest and need to make more progress on the print work first). 800 frames at 8 iterations = 6400 each seems a reasonable starting point for prints and for a number of layers in the moving image.
I did a little work to see how 1000 (before I determined that would not likely fit in GPU memory) compared to 250 in terms of print density. The following shows a test print (peak plus a little tail) of the previous 250 wave * 24 and the corresponding 1000 wave * 6 iteration plot. They are certainly looking compatible. I’ve also included a few details and am confident that 800 waves will look good. This will also allow the two works to match with not just the same density (as initially planned) but also the same number of layers. The only drawback is that the installation will be tuned to HW I don’t own, but that’s what tech riders are for.
The next step will be to tweak the upper frequency limit slightly, just to pull open the peak so that the balance between peak and non-peak areas will be easier to manage. I can then make test prints and see how the density looks and I’ll be back on track for making test prints.
Following from my last post, and looking up the specs of my printer’s hardware, I’m assuming a max (square) DPI of 1200. This gives me a benchmark for how the final print will look in terms of density. I had also erroneously reported that the line thickness did not seem to make a difference, but that was just an artifact of working with relatively low DPI settings; at 1200 DPI there is a significant difference between a 0.1 and 0.01 (let alone a 0.001) line. At 1200 DPI a 0.01 line ends up being 1/100th of a mm wide, which is fine enough! The following image is raster of my first initial test print just to narrow down the right opacity and line readability:
I’ve also included some full resolution details near that centre horizontal line for just the least (left) and most (right) opacity in this set:
If the print output matches what I see on my display, then the density at the right (alpha=0.23) is looking pretty good to me. Now lets see how this translates to the central peak, with the same line, resolution and opacity settings:
