In an ongoing collaboration with dr Frank Hirth‘s lab, a couple of months ago I have been invited to generate the code for a simulated insect navigating a virtual 2-dimensional world. This investigation aimed at testing whether a recently developed neural model of the central complex in insects (i.e. see related work here and here) could be exploited to guide purposeful navigation. [UPDATE: the paper has been accepted and will be soon published. You can find the abstract here: http://journal.frontiersin.org/article/10.3389/fnbeh.2017.00142/abstract]
The first step of this task was to graphically create both virtual arena and simulated insect. To test the system, I have first implemented a sort of test tube, where actions are not driven by a neural system, but rather selected randomly. This was used to test the movements of the agent (e.g. whether the obstacles were effective or ignored), simulated perception of cues in the arena (e.g. whether the visual field was properly coded) and in general all the graphical features related to both environment and agent.
The second step was to record the graphical output, so to be able to generate short videos of the behaviour of the simulated agent. There are several examples online: in this case, you can see the video is essentially defined in three blocks of code lines: “video initialization”, “frame capture” and “video recording” (see below, commented text).
I share here this preliminary code as a relatively simple example of how to produce a live stream of graphical outputs and record it to be seen later on. Again, I want to stress here that the agent in this example does not rely on a neural system to produce its behaviour. The actual study based on the use of the neural system is currently under review.
Video example of random navigation in the environment:
Source for the main file (at the end the zip file with the entire code).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
clear all close all format short g time=300; %rng(35) %% video initialization vd = VideoWriter('test_video', 'MPEG-4'); frm=1; %% ARENA x = linspace(0.0,4.0,100); y = zeros(length(x),1); fig=figure('Name', 'VIRTUAL FLY', 'Position', [100 200 1920 600]); %max: 1920-1080 arena_builder; ax = gca; h = hgtransform('Parent',ax); %hold on plot(x(1),y(1),'o','LineWidth', 20, 'color', 'k','MarkerFaceColor', 'k', 'Parent',h); %hold off %starting point and translation c1=ones(time,1); c2=ones(time,1); c1(1)=1+floor(10*(rand(1)-0.5)/3)/10; c2(1)=1+floor(10*(rand(1)-0.5)/3)/10; %display coordinates of the fly t6 = text(-.1,-.1,0, ['coordinates: %', num2str([c1(1) c2(1)])], 'VerticalAlignment','top','FontSize',14); direction=ones(time,1); face=zeros(time,4); face(1,:)=[1 0 0 0]; position=zeros(time,16); vision=zeros(time,4); subplot(2,3,[2 3]) colormap(bone) axis([0 time -0.1 16.1]) title('Where in the arena') subplot(2,3,5) %cue=animatedline('LineWidth', 2.5); axis([0 time 0 5]) title('Perceived cue') subplot(2,3,6) %bod= animatedline('LineWidth', 2.5); axis([0 time 0 5]) title('Body orientation') for k = 2:time %action seleztion if floor(k/5)==k/5 disp(k); direction(k)=randi(3,1); else direction(k)=1; end behaviour_control [c1, c2]=arena_border(c1, c2, k); position=position_control(position, c1, c2, k); vision=vision_control(vision, position, face, k); m1 = makehgtform('translate',c1(k),c2(k),0); h.Matrix = m1; t1; t6.String = num2str([c1(k) c2(k)]); %addpoints(bod,k,face(k)); %addpoints(cue,k,vision(k)); subplot(2,3,[2 3]) imagesc(position') title('Where in the arena') subplot(2,3,5) imagesc(vision') title('Perceived cue') subplot(2,3,6) imagesc(face') title('Body orientation') drawnow wait(1) %frame capture mvm(frm) = getframe(fig); frm=frm+1; end %% video recording open(vd) writeVideo(vd,mvm) close(vd) %sources %https://www.mathworks.com/help/matlab/ref/animatedline.html %https://www.mathworks.com/help/matlab/creating_plots/move-group-of-objects-along-line.html |
To download the archive with all the files for the entire code: fly_game