Q:

Is it possible to access the key attribute value when handling an event from template for each - LWC

Can anyone please help me on this?
 <div class="slds-grid slds-grid--vertical slds-navigation-list--vertical">
                                <ul>
                                    <template for:each={wiredStudentDataRefresh.data} for:item="acc" for:index="index">
                                        <li key={acc.Id}>
                                            <a href="javascript:void(0);" 
                                            class="slds-navigation-list--vertical__action slds-text-link--reset"
                                                aria-describedby="entity-header"  data-item={acc.FirstName} onclick={handleGetStudents}>{acc.FirstName}</a>
                                        </li>

                                    </template>
                                </ul>
                            </div>

handleGetStudents(event){
alert('Hello' + event.target.dataset.key);
}

A:

You are already utilising data-item so the access in JavaScript is
event.target.getAttribute("data-item");

This will return FirstName in your case. You can add multiple data- items to store whatever information, for example:
<a [...] data-record-id={acc.Id}>

