1 line
No EOL
17 KiB
JavaScript
1 line
No EOL
17 KiB
JavaScript
'use strict';(function(){const t={cache:!0};t.doc={id:"id",field:["title","content"],store:["title","href","section"]};const e=FlexSearch.create("balance",t);window.bookSearchIndex=e,e.add({id:0,href:"/docs/developer/",title:"Developer Documentation",section:"Docs",content:"This is the developer documentation. (Work-in-progress)\n"}),e.add({id:1,href:"/docs/user/",title:"User Documentation",section:"Docs",content:"This is the user documentation. (Work-in-progress)\n"}),e.add({id:2,href:"/posts/week-9/",title:"Week 9",section:"Blog",content:"This week I worked on getting focus detection working. I implemented basic laplatian blur detection1 and fast fourier blur detection2.\nFinding the threshold for both can be a challenge\nGeneral pipeline # I continued to develop a general pipeline to fit all the filters in\n https://pyimagesearch.com/2015/09/07/blur-detection-with-opencv/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n https://pyimagesearch.com/2020/06/15/opencv-fast-fourier-transform-fft-for-blur-detection-in-images-and-video-streams/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:3,href:"/posts/week-8/",title:"Week 8",section:"Blog",content:"Monday # Towards the end of last week ( Week 7), I managed to refactor my code in order to make it more portable. This allowed me to train my model on different machines. I ran my training script on the uni\u0026rsquo;s GPU compute successfully for 20 epochs. The next stage was to train it for longer and analyse the results. On the Monday morning I adjusted the parameters of my training script to train for 2000 epochs instead.\nTuesday # Tuesday afternoon the training had finished and I had a model that was trained on 2000 epochs. This gave me a day to analyse the results and do some rough predictions before my mid-project demo on the Wednesday.\nTraining and validation loss graphs # As we can see from the 2000 Epochs graph, the loss seems to plateau at around 60 epochs. The training loss seems to even out with more accuracy than the validation loss. This means that our data isn\u0026rsquo;t fully learning what I want it to. Also it\u0026rsquo;s overfitting slightly as it\u0026rsquo;s better at predicting the training set than the validation set. The variance in the validation set shows that the features it\u0026rsquo;s decided to learn aren\u0026rsquo;t the right features to confidently predict aesthetics in this dataset.\nFor the rest of the day I worked on my prediction script so I could use the model to predict new pictures. I also worked on my architecture diagrams and slides for the mid-project demo.\n Due to the nature of how I processed my images (resizing them to 32x32 and then saving them to a tensor then saving them to disk), my prediction script also displayed those down-sized images. This may have also effected the performance of the model.\nWednesday # I spent most of Wednesday morning finishing my slides, diagrams and making example predictions using the prediction script.\n Rest of week # I spent the rest of the week looking at the project\u0026rsquo;s overall pipeline including the non-machine learning filtering. I also started to implement basic focus detection by looking at blur detection using the Laplacian operator1.\n https://pyimagesearch.com/2015/09/07/blur-detection-with-opencv/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:4,href:"/posts/week-7/",title:"Week 7",section:"Blog",content:"Now that I had successfully run my model without any runtime errors, the next step this week was finding some GPU compute so I can train my model on much more powerful hardware to accelerate the training.\nMy first idea was to use cloud computing. There are machine learning specific cloud technologies, but I didn\u0026rsquo;t want to use these as I didn\u0026rsquo;t want my code to be dependent on the specific ways cloud platforms want the code in. Instead, I wanted to get a general VM with an attached GPU where I could run my workloads manually. I had already written docker images that contained all the depencies of my code that I could deploy to these VMs to ensure a reproducible and portable environment.\nFirst place I looked was Linode. Although, after I contacted their support about it they said I needed at least $100 of transactions on my account in order to request access to their GPU instances. They also noted I could make a deposit of $100 to start using them straight away. I wasn\u0026rsquo;t sure if my model was going to use up $100 to train yet so I didn\u0026rsquo;t want to risk it.\nI then looked to Microsoft\u0026rsquo;s Azure. I had used Azure during my industrial year and had previously passed the fundamentals and associate administrator exams for the platform so felt fairly confident in using it for my project. I ran into some issues I couldn\u0026rsquo;t quite explain at the start of the week. For some reason no services were available for me to use. I couldn\u0026rsquo;t use any VMs, create any networks or drives etc\u0026hellip; Turns out I noticed that my UK account was defaulting to trying to create resources in the US which I didn\u0026rsquo;t have access to. So I had to manually set my location to the UK in order to create any resources.\nWhile I was trying to work out the issue with Azure, I looked at using GCP. GCP automatically sets new accounts to have a quota of 0 for GPUs. Meaning you can\u0026rsquo;t attach one to any VM. You can increase the quota which requires getting in contact with customer support. Within 10 minutes I got a response and my quota was increases by 1. I wrote Terraform IaC (Infrastructure as Code) to automatically spin up and spin down cloud resources quickly.\nAt this point I realised my code wasn\u0026rsquo;t very portable as it included hard-coded absolute paths among other things. I refactored a lot of my code in order to run it on any machine. I was also granted access to the university\u0026rsquo;s GPU compute servers which allowed me to train my model without paying for cloud fees. To make sure my refactoring worked and that the GPU was actually being utilised during training, I rain the training script on the uni\u0026rsquo;s GPU compute server for 20 epochs. It successfully finished in ~13 minutes.\n"}),e.add({id:5,href:"/posts/week-6/",title:"Week 6",section:"Blog",content:"This week I finished programming the basic CNN model using transfer learning. I decided to train it for 20 epochs to make sure there weren\u0026rsquo;t any runtime errors in my code. As I don\u0026rsquo;t own an Nvidia GPU (I have an AMD GPU), I couldn\u0026rsquo;t make use of the pytorch version that utilised CUDA to speed up processing. There is a RocM version of pytorch for AMD GPUs1 but RocM isn\u0026rsquo;t as mature as CUDA and only officially supports a small subset of Linux distributions. Therefore, for this model I trained on the COU for only 20 epochs just to make sure it ran successfully before trying to find some Nvidia compute.\nTransfer learning model architecture # Full # Simplified # Validation and train loss over 20 epochs # https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:6,href:"/posts/week-5/",title:"Week 5",section:"Blog",content:"Starting to write a CNN # This week I started to implement what I had learnt about CNNs in Week 4. At this point I hadn\u0026rsquo;t designed a CNN architecture to implement, instead I wanted to have a running model regardless of performance just to see if I could implement one and understand it.\nHalf way through implementation, I decided to look back at the existing research papers on judging aesthetic judgement to see which aspects of their system and CNN were important to the task. While reading I saw that nearly all research papers on the topic used transfer learning instead of creating their own CNNs from scratch. At this point I stopped writing a CNN from scratch and decided to solve my problem using transfer learning instead. I followed a guide1 on implementing transfer learning in pytorch and another guide23 on creating a regressive CNN model.\n A Rosebrock. \u0026ldquo;PyTorch: Transfer Learning and Image Classification\u0026rdquo;. pyimagesearch.com. https://pyimagesearch.com/2021/10/11/pytorch-transfer-learning-and-image-classification/ (accessed Mar. 1, 2022.)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n A Rosebrock. \u0026ldquo;Regression with Keras\u0026rdquo;. pyimagesearch.com. https://pyimagesearch.com/2019/01/21/regression-with-keras/ (accessed Feb. 22, 2022.)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n A Rosebrock. \u0026ldquo;Keras, Regression, and CNNs\u0026rdquo;. pyimagesearch.com. https://pyimagesearch.com/2019/01/28/keras-regression-and-cnns/ (accessed Feb. 22, 2022.)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:7,href:"/posts/week-4/",title:"Week 4",section:"Blog",content:"This week I did some research into how to build a CNN from scratch, including the different type of layers, loss functions, learning rates, epochs and other core concepts.12\nI also set up and created this blog with Hugo to document my progress and setup Woodpecker CI to do continuous testing and integration.\n deeplizard. \u0026ldquo;Convolutional Neural Networks (CNNs) explained.\u0026rdquo; (Dec. 9, 2017). Accessed: Feb. 22, 2022. [Online Video]. Available: https://youtube.com/watch?v=YRhxdVk_sIs\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n A Rosebrock. \u0026ldquo;PyTorch: Training your first Convolutional Neural Network (CNN)\u0026rdquo;. pyimagesearch.com. https://pyimagesearch.com/2021/07/19/pytorch-training-your-first-convolutional-neural-network-cnn/ (accessed Feb. 22, 2022.)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:8,href:"/posts/week-3/",title:"Week 3",section:"Blog",content:"Filming footage # At the start of the week I went into town to film some practise footage to work with later (up until this point I had been experimenting with footage limited by my bedroom walls). I took some basic vertical and horizontal footage of the town - no nature or breach footage yet.\nGaining more useful information # I used the footage I had recorded at the start of the week and revised my \u0026ldquo;filesize\u0026rdquo; code. I made a new function order_frames_by_filesize() which orders the frames by the filesize and prints the name of size of each frame in order.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 def order_frames_by_filesize(): \u0026#34;\u0026#34;\u0026#34;Order the frames by filesize and print the filenames and their sizes\u0026#34;\u0026#34;\u0026#34; frames = os.listdir(outputfolder) # sort the frames by their filesize frames = sorted(frames, key = lambda x: os.stat(os.path.join(outputfolder, x)).st_size, reverse = True) # print every frame and it\u0026#39;s size in a human readable format for frame in frames: filesize = os.stat(os.path.join(outputfolder, frame)).st_size if filesize \u0026gt; 1024: filesize = filesize / 1024 print(frame + \u0026#34;: \u0026#34; + str(filesize) + \u0026#34; KB\u0026#34;) else: print(frame + \u0026#34;: \u0026#34; + str(filesize)) Analysing datasets # There\u0026rsquo;s been a few datasets that I\u0026rsquo;ve previously looked at which could be useful to use for this project. I wasn\u0026rsquo;t sure how they would perform with the data I was expecting to use. AVA1 was trained on data that had been photographed under ideal conditions by profesionals. This won\u0026rsquo;t reflect the frames extracted by the footage in this project\u0026rsquo;s use case. I wasn\u0026rsquo;t sure if this distinction was significant enough to effect the results of a trained model. I had previously found a project2 which had models pretrained using AVA1 I could use to predict an aesthetic value from images I provided.\n1:1 Weekly meeting # We discussed that during week 4 I should be looking at implementing CNNs as training might take a while and therefore should be a priority. Look at what other people are doing with aesthetic analysis to get an idea on how the code works. Attempt to get a basic CNN working. N. Murray, L. Marchesotti and F. Perronnin, \u0026ldquo;AVA: A large-scale database for aesthetic visual analysis,\u0026rdquo; 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2408-2415, doi: 10.1109/CVPR.2012.6247954.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n Image Quality Assessment: https://github.com/idealo/image-quality-assessment - Idealo\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "}),e.add({id:9,href:"/posts/week-2/",title:"Week 2",section:"Blog",content:"This week I set up my repositories and starting writing some basic code.\nSet up # Before starting any coding, I wanted to set up my remote git repositories. I had already decided I wanted the project mirrored over two remote git repositories from different providers as a safety precaution. My intial plan was to use the university\u0026rsquo;s GitLab instance but as it\u0026rsquo;s recently been moved behind the firewall it would have made mirroring quite difficult. Instead, I decided to use my personal Gitea instance and mirror it to my personal account on the official GitLab instance.\n Gitea → GitLab\nGitea will periodically push to GitLab.\nCode # Towards the end of the week I put together some simple code to reduce the frame set. I used the OpenCV Python module to export the frames of a given video to an output/ folder. The path of this folder is also stored in the outputfolder variable we see used on line 7.\nAlthough this code is a first step at reducing the frames set - it doesn\u0026rsquo;t give us much feedback about the frames that have been deleted or kept.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 def display_file_sizes(): \u0026#34;\u0026#34;\u0026#34;Calculates the average frame filesize and deletes the frames lower than the average \u0026#34;\u0026#34;\u0026#34; # create an array storing the filesize of each frame filesizes = [] for filename in os.listdir(outputfolder): filepath = outputfolder + \u0026#34;/\u0026#34; + filename filesize = os.path.getsize(filepath) print(filepath + \u0026#34;: \u0026#34; + str(filesize)) filesizes.append(filesize) # work out average average = sum(filesizes)/len(filesizes) print (\u0026#34;Average is: \u0026#34; + str(average)) # delete files below average count = 0 for filename in os.listdir(outputfolder): filepath = outputfolder + \u0026#34;/\u0026#34; + filename if filesizes[count] \u0026lt; average: # print(filepath + \u0026#34;: \u0026#34; + str(filesizes[count])) os.remove(filepath) count += 1 I didn\u0026rsquo;t end up doing as much as I would have liked this week. I found myself preoccupied in other areas of my life and not putting enough emphasis on this project.\n1:1 Weekly meeting # As I didn\u0026rsquo;t do as much as I would like to have done there wasn\u0026rsquo;t much to discuss on my part I needed to produce more code to have some basic functionality. Discussed my concerns with a machine learning approach "}),e.add({id:10,href:"/posts/week-1/",title:"Week 1",section:"Blog",content:"This week is the first week of the project. I researched academic papers, existing code and dataset relating to the topic of determining aesthetics.\nPapers # Photo Aesthetics Analysis via DCNN Feature Encoding1 - Predicting aesthetic performance using a bespoke CNN solution\n AVA: A large-scale database for aesthetic visual analysis2 - Making of an aestehtic visual analysis dataset\nCode # Image Quality Assessment - Convolutional Neural Networks to predict the aesthetic and technical quality of images.\nDatasets # AADB\nAVA: https://github.com/imfing/ava_downloader, https://github.com/ylogx/aesthetics/tree/master/data/ava\nProject idea from research # Based on the research, I decided a machine learning approach would result in higher quality outputs. Although, I was slightly concerned that following a deep-learning would limit interesting discussion in my report.\nThe idea was to create a program that can take a video, break it down into frames and use a trained CNN to predict the most aesthetic frames and return them to the user.\nWeekly 1:1 meeting # During the meeting I mentioned my concerns following a deep learning approach. Although this approach might provide quality results, it doesn\u0026rsquo;t provide much room to discuss or develop interesting solutions. Instead, as Hannah put, it mostly depends on throwing the problem at powerful hardware to get the best output which doesn\u0026rsquo;t make for an interesting project. Hannah suggested I take a hybrid approach where I could use deep-learning for the last step in the pipeline, depending more on conventional engineering techniques to reduce the input data before passing it to the deep-learning stage.\nShe mentioned \u0026lsquo;dumb\u0026rsquo; ways in which I could reduce the set of input frames:\n Comparing file sizes and removing the small ones (might infer single colour images / less complex images) Fourier frequency analysis Brightness and contrast analysis H. -J. Lee, K. -S. Hong, H. Kang and S. Lee, \u0026ldquo;Photo Aesthetics Analysis via DCNN Feature Encoding,\u0026rdquo; in IEEE Transactions on Multimedia, vol. 20, no. 8, pp. 1921-1932, Aug. 2017, doi: 10.1109/TMM.2017.2687759.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n N. Murray, L. Marchesotti and F. Perronnin, \u0026ldquo;AVA: A large-scale database for aesthetic visual analysis,\u0026rdquo; 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2408-2415, doi: 10.1109/CVPR.2012.6247954.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n "})})() |