Shelf Monitoring IoT Retail

Shelf Monitoring IoT Retail — Topics for IoT Students

AttenFace: A Real Time Attendance System Using

Face Recognition Ashwin Rao International Institute of Information Technology, Hyderabad ashwin.rao@students.iiit.ac.in

Abstract—The current approach to marking attendance in attendance still share a common problem: there is no way to colleges is tedious and time consuming. I propose AttenFace, a ensure that students sit in class through its entire duration. A standalone system to analyze, track and grant attendance in real arXiv:2211.07582v1 [cs.CV] 14 Nov 2022

student can leave class immediately after attendance, or enter time using face recognition. Using snapshots of class from live camera feed, the system identifies students and marks them as class just before attendance. present in a class based on their presence in multiple snapshots AttenFace’s novel snapshot technique of face recognition taken throughout the class duration. Face recognition for each (described in Sec.

III) solves all these problems, including class is performed independently and in parallel, ensuring that the issue of proxy attendance which is common in colleges. the system scales with number of concurrent classes. Further, Students no longer have to manually give attendance, as their the separation of the face recognition server from the back- end server for attendance calculation allows the face recogni- presence is automatically recognized by a camera.

Since the tion module to be integrated with existing attendance tracking camera is recording at all times, it is easy to capture how long software like Moodle. The face recognition algorithm runs at 10 a student remains in class. Final attendance can be given only minute intervals on classroom snapshots, significantly reducing if the student remained in class above a certain threshold of computation compared to direct processing of live camera feed. time, which can be decided by the professor teaching the class.

This method also provides students the flexibility to leave class for a short duration (such as for a phone call) without losing Further, the system includes an easy-to-use portal for students attendance for that class. Attendance is granted to a student to check their attendance for any class and course, and for if he remains in class for a number of snapshots above a professors to override default attendance rules for a particular certain threshold.

The system is fully automatic and requires class or student if necessary. no professor intervention or any form of manual attendance or even camera set-up, since the back-end directly interfaces with in- II. R ELATED W ORK class cameras. AttenFace is a first-of-its-kind one-stop solution for face-recognition-enabled attendance in educational institutions The proposed attendance system requires three major tech- that prevents proxy, handling all aspects from students checking nologies to identify students: 1) object detection and localiza- attendance to professors deciding their own attendance policy, to tion, to identify which objects in the classroom are students, college administration enforcing default attendance rules. and where they are, 2) face detection, to identify which object Index Terms—real-time attendance, face recognition, software architecture, deep learning. is a face, and 3) face recognition, to map detected faces to corresponding students.

There is continuous research going on in these areas. YOLO [1] is a real-time object detection I. I NTRODUCTION algorithm which is being continuously improved upon since Attendance is a mandatory part of every class in colleges. its first introduction in 2016.

Haar Casading [2] identifies Often, there is a minimum attendance requirement for courses faces in a real-time video feed. FaceNet [3] is a notable face- taken by students. The simplest methods of taking attendance recognition technique, which uses a deep convolutional neural include roll-call or manually signing on a document.

These network with 22 layers trained via a triplet loss function to methods are tedious and waste time and do not take advan- directly output a 128-dimensional embedding. VGGFace2 [4] tage of technology in any way. Attendance will have to be is a dataset for training face recognition models taking into manually entered from the attendance sheet into the database. account pose and age.

Further, proxy attendance is easy, wherein a student gives At the heart of the proposed system is the face recognition attendance to another student, either by forging his signature algorithm. This problem has numerous approaches [5]. Princi- or calling out his name during roll-call.

A current solution to pal Component Analysis (PCA) extracts principal features to make attendance easier, and gaining popularity, is biometric recognize faces. Template matching involves comparing and attendance. The biometric machine can be connected to the matching patterns to recognize faces, usually through a neural database and update attendance automatically.

The problem network. Taking into account pose variations and liveliness of proxy is solved due to the uniqueness of thumb prints. detection can help make the system more robust. Liveliness However, this method still wastes time due to the students detection is the problem of differentiating between the face of needing to queue up for biometric attendance.

Passing the a live person and a photograph [6]. biometric machine around in class solves this issue, but can be In recent years, a number of face recognition based atten- disturbing. However, all the aforementioned methods of taking dance systems have been proposed. In [7], face recognition

along with Radio Frequency Identification (RFID) was used but more robust than existing solutions for face recognition, to detect authorized students and count the number of times a which do not run continuously during the whole class but student enters and leaves the classroom. In [8], students were rather involve a single verification before or after class. recognized through iris biometrics. The system automatically took attendance by capturing an image of the eye of each A.

Requirements student and searching for a match in the database. In [9], The requirements of the system are as follows: the Eigenface and Fisherface face recognition algorithms were 1) Functional requirements compared and used in a real-time attendance system. Eigenface • A login portal connected to the institute login, to be was found to perform better, with an accuracy of 70-90%. used by students, professors and administrators.

In [10], authors used Discrete Wavelet Transform (DWT) • A dashboard for students and professors to view and Discrete Cosine Transform (DCT) to extract features of attendance details for any class or course taken by students’ faces, followed by the Radial Basis Function (RBF) them. to recognize them. Their attendance system had an accuracy • An easy way for professors to make minor changes of about 82%. In [11], authors considered various real-time to attendance policy for a class or the entire course scenarios such as lighting and pose of students.

A 3D approach directly from the dashboard without needing to go to recognize faces for attendance was put forward in [12]. through the administration. There are a number of attendance systems available on the • An option on the dashboard for the administration market which use face recognition for identification. Most of to manually override faulty attendance results. these, however, are time and attendance systems, which are used for one-time close-up identification of a single person at 2) Non-functional requirements a time.

For example, Truein [13] is a touchless face recognition • The system should be able to access required student system used to manage employee attendance in the workplace. information, course information and class informa- iFace [14] provides face recognition capabilities through a tion from the institute database. mobile app, useful in work-from-home scenarios. There is • Portal to view attendance should be cross-platform currently no product in the market aimed at real-time face with emphasis on mobile friendliness. recognition for attendance capture in schools and colleges that • The face recognition algorithm should be able to provides a single unified portal to track attendance and modify recognize faces in real-time, without much compu- attendance policy, which can also integrate with existing tational overhead. attendance management systems.

The underlying technology • Multiple instances of the face recognition algorithm of face recognition, however, is the same, and can be adapted should be able to run in parallel, since there can be to suit the proposed system. multiple classes going on at any given time. III. S YSTEM ARCHITECTURE B.

Use Cases The proposed system uses face recognition to automatically handle attendance of students. Taking the actual face recogni- tion algorithm as a black box, the system decides attendance in the following manner: • Recording begins at the start of the class. The start time of the class and the room number are available via the database. • A snapshot of the class is taken every 10 minutes.

Using the snapshot, the face recognition algorithm recognizes students and marks them as present in that 10 minute block of time. • A student will be marked present for a class if he is present in at least ’n’ snapshots. This threshold can be decided by the professor. This allows a student to leave the class in between in case of an emergency without losing attendance.

AttenFace’s snapshot model provides a method to contin- uously track attendance throughout the class duration while avoiding the computationally expensive process of face recog- nition on live video. Simultaneously, it ensures that students Fig. 1. UML use case diagram. must remain in class for a minimum amount of time to receive attendance, solving the issue of students leaving class after 1) A user (student, professor or admin) should be able to manual attendance.

This makes the system equally efficient log in using their college credentials.

2) A student should be able to view his attendance for any information will be displayed for a student: a) total class of any course he has taken. attendance received so far in a particular course, b) 3) A student should be able to view his attendance (0 or attendance received in any class of any course he has 1) for a class as soon as it ends. registered for, c) the total ”blocks” attended in any 4) A student should be able to see how many ”blocks” class to justify the attendance for that class, and d) the in a class he attended (as explained in the previous threshold to determine whether attendance is obtained or section) and whether this is above the threshold set by not in a particular class.

The following information will the professor, hence justifying the attendance obtained be displayed for a professor: a) total attendance for any for that class. class in any course taught by him, and b) the threshold 5) A student should be able to view the total number of to determine attendance (which he can edit). classes he has attended in a course so far, and the number of classes he is allowed to miss before his total attendance for that course drops below the course’s requirements. 6) A professor should be able to view the total attendance of a class as soon as it ends. 7) A professor should be able to change the threshold determining for how many blocks a student must be present in the class to get attendance.

This can be changed for a specific class or for all classes in the course. This gives the professor the power to make attendance lenient or optional on a specific day, without having to go through the college administration. 8) A professor should be able to change the room number of the class before it starts, in case of any sudden events to ensure that attendance will still be taken by activating the camera in the new room. 9) An administrator should be able to directly change the attendance of a student in a particular class if required, overriding the automated attendance. 10) An administrator should be able to change the room number of the entire course, in case of any clashes with other classes or events.

Fig. 3. Sample interface for students. C.

System Architecture

Fig. 2. System architecture. Fig. 4.

Sample interface for professors. The system can be divided into the following modules: 1) Front-End (mobile and web application): All users inter- 2) Back-End: The back-end handles all interactions with act with the system through the frontend. The following the database including accessing student images for the

face recognition algorithm, displaying all relevant data on the front-end, and standard CRUD operations. It also performs the following calculations: a) total attendance of a student in a course given his attendance in all classes of the course so far, and b) attendance of a student in a class of a course given his attendance in each block of time in that class. The back-end acts as a bridge to send data to the face recognition server, which cannot access the database directly. 3) Face Recognition Model Server: The computationally heavy operations for face recognition will be performed here.

Each ongoing class will have its own thread of computation associated with it, which obtains live feed directly from the concerned camera, and communicates with the back-end server for attendance calculation. Each thread of the face recognition server receives the following data from the back-end: a) Images of all students attending the class, b) start time of the class, c) end time of the class, and d) the camera ID to activate and obtain live feed from.

The interaction diagram of the face recognition server with the back-end and camera is shown in Fig. 6. The direct communication between the back-end and camera is an integral component of making the system standalone and fully automatic compared to existing solutions. Fig. 5.

UML class diagram. 4) Database: It stores information regarding students (such as images for recognition and the courses they have registered for), courses (such as room number and the corresponding camera ID), and professors (such as courses they are teaching). Fig. 8 shows a simplified schema of the database.

D. University Classroom: A Practical Example

A walkthrough of using AttenFace for a camera-enabled classroom in a university would be as follows: • The system establishes a connection with the camera 5 minutes before class starts. • The professor, if he wishes to do so, logs into the portal and change attendance requirements for the current class (see Fig. 4). • Starting from the beginning of class, and until class ends, a snapshot of the classroom is sent to the back- end and subsequently, the face recognition server, every 10 minutes.

Students are identified and their presence is marked in that 10-minute block of time. • After class ends, students can log in to the portal and immediately view their attendance status for that class (see Fig 3). E. Extensibility and Ease of Integration The proposed system is modular.

The face recognition server, in particular, is a standalone module, which plays no role in the actual attendance policy of the professor or institute. Given pictures of students as input, all recognition- Fig. 6. UML 2.0 sequence diagram showing interactions between the back- based calculations are done within the face recognition module end server, database and face recognition server. and results for each student (whether present or not for that

and at the same time grants a certain amount of leniency in the attendance calculation, as decided by the professor. There is always scope for improvement. Face recognition techniques are not completely accurate, and the system may sometimes be unable to identify students, or recognize them incorrectly.

External factors such as classroom lighting and position of students faces may have an effect on the accuracy of the face recognition algorithm. As new research leads to better performing face recognition algorithms which are more robust and adaptable to varying situations, the proposed system benefits. R EFERENCES [1] Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, ”You Fig. 7.

UML sequence diagram showing interaction between user and front- Only Look Once: Unified, Real-Time Object Detection”, 2016. end. [2] Paul Viola, Michael Jones, ”Rapid Object Detection using a Boosted Cascade of Simple Features”, 2001. [3] Florian Schroff, Dmitry Kalenichenko, James Philbin, ”FaceNet: A Unified Embedding for Face Recognition and Clustering”, 2015. [4] Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, Andrew Zisserman, ”VGGFace2: A dataset for recognising faces across pose and age”, 2017. [5] Face recognition algorithms: http://www.ehu.eus/ccwintco/uploads/e/eb/PFC- IonMarques.pdf [6] Saptarshi Chakraborty, Dhrubajyoti Das, An overview of face liveliness detection, International Journal on Information Theory (IJIT), Vol. 3, No. 2, April 2014. [7] Akbar, Md Sajid, et al. ”Face Recognition and RFID Verified Atten- dance System”, International Conference on Computing, Electronics & Communications Engineering (iCCECE), IEEE, 2018. [8] Okokpujie, Kennedy O., et al. ”Design and implementation of a stu- dent attendance system using iris biometric recognition”, International Conference on Computational Science and Computational Intelligence (CSCI), IEEE, 2017. [9] Siswanto, Adrian Rhesa Septian, Anto Satriyo Nugroho, Maulahikmah Galinium, ”Implementation of face recognition algorithm for biometrics based time attendance system”, International Conference on ICT For Smart Society (ICISS), IEEE, 2014. [10] Lukas, Samuel et al. ”Student attendance system in classroom using face recognition technique”, International Conference on Information and Communication Technology Convergence (ICTC), IEEE, 2016. [11] Rathod, Hemantkumar, et al. ”Automated attendance system using machine learning approach”, International Conference on Nascent Tech- Fig. 8.

Simplified database design. nologies in Engineering (ICNTE), IEEE, 2017. [12] MuthuKalyani, K., A. VeeraMuthu. ”Smart application for AMS using face recognition”, Computer Science & Engineering 3.5 (2013): 13. block of time) are returned to the back-end server, which [13] Touchless Face recognition employee attendance and visitor manage- ment: https://truein.com/ handles attendance calculation using this data. Due to its [14] https://time-attendance.bioenabletech.com/software-solutions/touch- modular nature, the system can be easily integrated into less-face-recognition-based-employee-attendance existing college portals.

For example, integrating the proposed real-time attendance system with moodle is straightforward. The front-end, institute login and interaction with the college database are already handled by moodle. The only part of the system which needs to be integrated is a custom back- end script which interacts with the face recognition server and performs the desired calculations, and the face recognition server itself.

The attendance data can then be made available to moodle to display on the frontend.

IV. C ONCLUSION AND FUTURE WORK

This paper proposes a new method to analyze and grant attendance in real time using face recognition. Attendance in each class is determined automatically with no human effort. The system ensures that a student must stay in class for at least a certain amount of time to be marked present,