Creating the code for this app was an journey, notably diving into the complexities of digicam views and film classification. No matter encountering obstacles alongside the way in which by which, the help of buddies and collaborative brainstorming lessons helped me uncover my method. Each downside turned an opportunity for finding out and enchancment, as I navigated by the use of trial and error to raised understand these utilized sciences. Finally, the arduous work paid off, and I’m happy with what I achieved. It was a journey full of ups and downs, nevertheless the sense of accomplishment made all of it worthwhile.
Proper right here comes primarily probably the most tough part of this app enchancment and I’ll let you understand why
Making a model with CoreML for this enterprise was pretty tough. Initially, I aimed to classify poses into two lessons: “flawed pose,” which included information the place the legs and arms have been too far apart, and “correct pose,” which included information the place the legs and arms have been shut collectively, notably hand-to-hand and leg-to-leg.
However, I encountered a problem the place sitting was incorrectly labeled as a “correct pose.” To take care of this, I added a third class, “others,” which included pictures of people sitting and standing. Sadly, this led to a unique draw back: the model labeled the suitable pose as “others” with 100% confidence, inflicting considerable frustration.
As soon as we’re misplaced, we have now to ask others
After further investigation and dialogue with buddies, I noticed the knowledge needed a further generalized pattern fairly than detailed specifics. Consequently, I redefined the programs: “correct pose” notably contained the Tadasana yoga pose, whereas “flawed pose” included a mix of various yoga poses.
Lastly, this revised model labored as supposed, exactly predicting the poses as deliberate.
All through this course of, I encountered a lot of challenges when attempting to put the digicam view and completely different UI elements in a 60:40 ratio, with the UI elements on the left. The primary problem was that the digicam view wouldn’t accurately fill the right side of the online web page. Given that app is designed to be used in panorama mode, the digicam view moreover displayed in panorama format. However, I needed the digicam view to be in portrait orientation.
I experimented with many different choices, nevertheless none of them labored. The digicam view each remained in panorama mode or might very properly be compelled into portrait mode nevertheless displayed a rotated view, as if the digicam was being utilized in a portrait app.
It’s tough and usually seems not doable, nevertheless we should always on a regular basis uncover an answer to improvise
Lastly, after further investigation, I discovered that the digicam view may very well be set to a personalized measurement, nevertheless this ends in it being zoomed in. Although this zooming impression was not supreme, it allowed me to swimsuit the digicam view into the required 60:40 construction ratio. By adjusting the digicam view’s measurement and accepting the zoomed-in present, I was ready to acquire a helpful and visually attention-grabbing construction. This methodology ensured that the digicam view displayed precisely alongside the other UI elements, providing a larger client experience.
Proper right here is the code that i used for make the digicam view’s measurement may very well be customise with consequence it’s going to be zoomed in.
func setupCamera() {
captureSession = AVCaptureSession()
guard let captureDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, place: .entrance),
let enter = try? AVCaptureDeviceInput(system: captureDevice) else { return }captureSession.addInput(enter)
let videoOutput = AVCaptureVideoDataOutput()
videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "videoQueue"))
captureSession.addOutput(videoOutput)
let previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer.videoGravity = .resizeAspectFill
previewLayer.connection?.videoOrientation = .landscapeRight
previewLayer.physique = previewFrame
view.layer.addSublayer(previewLayer)
captureSession.startRunning()
}
To mix the ML model with the view, I configured the conceitedness stage to 0.8. This means the poseStatus variable will solely substitute and be displayed on the left side of the view if the model’s confidence is as a minimum 0.8. By implementing this threshold, the poseStatus will not change for lower confidence ranges, resulting in further safe and proper strategies for the buyer.
OnePose Yoga is now a completely helpful app, though it does have a minor bug. For individuals who’re occupied with exploring the code behind the occasion of this app, be glad to check out the GitHub repository!
