We consider the problem of accurate sparse fine-tuning of large language models (LLMs), that is, fine-tuning pretrained LLMs on specialized tasks, while inducing sparsity in their weights. On the accuracy side, we observe that standard loss-based fine-tuning may fail to recover accuracy, especially at high sparsities. To address this, we perform a detailed study of distillation-type losses, determining an L2-based distillation approach we term SquareHead which enables accurate recovery even at higher sparsities, across all model types. On the practical efficiency side, we show that sparse LLMs can be executed with speedups by taking advantage of sparsity, for both CPU and GPU runtimes. While the standard approach is to leverage sparsity for computational reduction, we observe that in the case of memory-bound LLMs sparsity can also be leveraged for reducing memory bandwidth. We exhibit end-to-end results showing speedups due to sparsity, while recovering accuracy, on T5 (language translation), Whisper (speech translation), and open GPT-type (MPT for text generation). For MPT text generation, we show for the first time that sparse fine-tuning can reach 75% sparsity without accuracy drops, provide notable end-to-end speedups for both CPU and GPU inference, and highlight that sparsity is also compatible with quantization approaches. Models and software for reproducing our results are provided in Section 6.

Boravak u zatvorskom okruţenju predstavlja specifiĉno iskustvo za većinu pravosnaţno osuĊenih osoba. Iako se svrha kaţnjavanja ogleda u ostvarivanju ciljeva specijalne i generalne prevencije, treba imati u vidu da prilagoĊavanje na posebne uslove ţivota kao što su ograniĉenje slobode kretanja i upravljanja ustaljenim ţivotnim navikama predstavljaju sloţenu novonastalu situaciju koja utiĉe na opšte ali i mentalno zdravlje zatvorenika. Dalje, odlazak jedne osobe na izdrţavanje kazne zatvora, u velikom broju sluĉajeva, direktno utiĉe na brojne aspekte ţivota samog zatvorenika ali i ĉlanova njegove porodice a moţe imati dalekoseţne posljedice i nakon izlaska iz zatvorskih rešetki. Da li zatvor utiĉe na svijest zatvorenika o zdravim ţivotnim navikama i vrijednostima? Kako zatvor utiĉe na sposobnost zatvorenika da promjeni svoje ponašanje? Kakva prava u zatvoru imaju zatvorenici? Da li izlaze kao bolje ili lošije osobe? Da li zatvorske kazne utiĉu na zdravlje porodice i da li proces rehabilitacije kod zatvorenika uspije baš svaki put? Sve su to pitanja koja će biti obraĊena u okviru teme ovog referata.

Rad ukazuje na izazove odrastanja djece u digitalnom vremenu koje nudi brojne pozitivne mogućnosti, ali i opasnosti vezane za njihov zdrav razvoj. Cilj rada je ukazati kako savremeni mediji utiču na vaspitanje i oblikuju razliĉite dimenzije ţivota djece i adolescenata, njihov identitet i odnose s roditeljima i vršnjacima. U radu se kroz teorijsku analizu i rezultate istraţivanja daje odgovor na tri grupe pitanja: Koji su izazovi odrastanja djece u digitalnom vremenu? Kakav je uticaj digitalnih tehnologija na porodicu? Kakav je odnos roditeljstva i digitalnog učenja? Polazeći od toga da su ključni zadaci odrastanja razvoj identiteta, autonomije i intimnosti, odnosno stvaranje prijateljstava i odnosa, sugeriše se potreba analiza uticaja moderne tehnologije na sve navedene aspekte. U radu su saţete spoznaje o tome kako moderni mediji oblikuju razliĉite dimenzije ţivota djece i adolescenata, pritom uvažavajući njihove mogućnosti i prednosti. Digitalni život može podržavati ili ometati razvoj identiteta. Na osnovu teorijske analize i rezultata istraživanja su razmatrani uticaji medija na kvalitet odnosa s vršnjacima, te ukazano i na negativne pojave (internet zavisnost, cyberbullying). Uz odnose su analizirani i drugi konstrakti kako koristiti prednosti koje nude mediji i smanjiti loše vaspitne uticaje, te kako roditelji mogu pomoći djeci da se uspješnije nose sa izazovima digitalnog vremena. Rezultati istraživanja impliciraju važnost obezbjeđivanja podrške roditeljima dok poduzimaju korake prema ostvarenju vizija za svoju djecu u digitalnom vremenu.

The demand for renewable resources in building construction is increasing, and wheat straw is an excellent option due to its superior environmental performance compared to traditional insulation materials. However, the hygrothermal properties of chopped wheat straw insulation have remained largely unexplored. At the moment, blown-in straw is only blown in vertically, although horizontal blowing would be more efficient depending on the situation. This study investigates the effect of different blowing techniques on the thermal properties of chopped wheat straw insulation, focusing on the difference between vertical and horizontal blowing techniques. In-situ-measured thermal conductivities were compared with design values used in energy balances. In addition, the long-term hygrothermal behavior of chopped wheat straw insulation treated with flame retardants was investigated. The methodology included heat flow plate measurements, needle probe measurements and laboratory measurements using the hot plate method. The results show that there is no significant difference in thermal performance between the blowing techniques. The measured thermal conductivities were lower than expected, challenging the current general normative moisture surcharge on the thermal conductivity of natural fiber insulation. The addition of the flame retardant had no noticeable effect on the hygrothermal properties of the chopped straw. Chopped wheat straw can be regarded as a highly ecological insulation material with great potential for the future.

Embedded systems find extensive applications in various aspects of daily life and their presence in the highly connected world of today is growing. With a large number of remote devices and sensors, there is a growing need for remote data processing. Microcontrollers, often integral components of such devices, can enable remote data manipulation through the implementation of a web server called embedded web servers. The implementation of a server on such a system poses challenges, such as limited resources, so it is crucial for the web server implementation to be compatible with the available resources. This paper focuses on the requirements for the implementation of a web server. Also, an embedded web server has been designed for specific application of monitoring air parameters. The implementation utilizes the LwIP stack and allows for remote access to measurement results within a local network. The paper will delve into the operational principles of both the web server and the entire system.

We observe a network scenario where parts of a secret are distributed among its nodes. Within the network, a group of attackers is actively trying to obtain the complete secret, while there is also the issue of some nodes malfunctioning or being absent. In this paper, we address this problem by employing graph multicoloring techniques, focusing on the case of a single attacker and varying numbers of malfunctioning nodes.

Introduction Basketball is a fast-paced intermittent team sport, wherein the players must exhibit different morphologies and fitness levels depending on their position. The aim of this study was to assess the differences in the athleticism of basketball players by playing positions and in its relations with body composition and anthropometric measures. It was hypothesized that calculated athleticism has stronger and better predictive relations with morphology compared to motoric tests alone. Methods 47 basketball players were divided into three groups according to playing position [guards (n = 14), forwards (n = 22), centers (n = 11)]. Body composition and anthropometrical measurements were done for all players. Athleticism was presented in terms of overall (OFS), jumping (JFS) and sprinting (SFS) fitness scores. Results Fitness scores were found to be more strongly related to body composition and anthropometry measurements than motoric tests alone. All three fitness scores were moderate to strongly correlated with skinfold thickness and fat percentage, while body height, fat-free mass, and lean body mass were positively correlated. Significant differences in athleticism fitness scores were found between guards and other groups. Forwards displayed superior athleticism in power and sprint abilities when compared to guards and centers. Multivariate logistic regression revealed that fat percentage, lean muscle mass, skinfold thickness, and arm span exhibited strong predictive capabilities in relation to athleticism scores. Conclusions Coaches and practitioners should be aware that athleticism includes a multitude of components, and they should use athleticism assessments before designing training regimens that are tailored to each position's unique needs.

Power system reliability is often threatened by lightning discharges causing the outages of the overhead lines. Surge arresters installed on high-voltage overhead transmission lines can improve their lightning performance. Studying lightning is a challenging task due to its stochastic nature and complex discharge mechanism. Lightning data based on lightning location systems has contributed to the recent improved understanding of lightning but there are still areas that require further research such as the parameters of the lightning current waveform. This paper describes the proposed line surge arresters' monitoring system for measuring lightning and line surge arrester currents as well as temperature and humidity. The proposed system is based on state-of-the-art measuring sensors, communication and information technologies. Software and hardware components of the system are described, including laboratory tests and results.

Introduction: Statins are lipid lowering medications, used for the prevention of cardiovascular diseases (CVD), but have shown to increase the risk of Type 2 diabetes mellitus. The aim of this study was to investigate the effects of high-potency statins, atorvastatin, and rosuvastatin on fasting glucose (FG) and hemoglobin A1c (HbA1c) in CVD patients. Methods: The case–control study included 123 patients from Tuzla Canton, Bosnia, and Herzegovina, with a diagnosis of CVD, treated in three health centers: Public Health Center Gračanica, Banovići, and Čelić. Of total patients, 84 were statin users (39 atorvastatin users and 45 rosuvastatin users) and 39 were not. Demographic data, diagnosis, and data of the therapy were taken from the medical records, as well as data of the FG and HbA1c, measured before or within 3 months of the statin therapy introduction. For the same patients, FG and HbA1c were also measured at least 3 months after the introduction of therapy. Results: Obtained results have shown a significant increase of FG in CVD patients on statin therapy in relation to control (p = 0.034). Comparing the diabetogenic effects of atrovastatin and rosuvastatin, it was found that the HbA1c in patients on atorvastatin therapy was significantly higher comparing to those on rosuvastatain therapy (p = 0.028). The FG was significantly increased (p = 0.027) after atrovastatin therapy. Similar results were obtained in diabetogenic CVD patients, where HbA1c on atorvastatin therapy was significantly higher comparing to HbA1c in those on rosuvastatain therapy (p = 0.039). A significant correlation was found between the increase in FG and HbA1c with the duration of atorvastatin therapy (p = 0.001 and p = 0.033), and between the increase in HbA1c and the duration of rosuvastatin therapy (p = 0.001). Conclusion: Long-term therapy with high-potency statins, atorvastatin, and rosuvastatin, may increase levels of FG and HbA1c in patients with CVD, where atorvastatin shows more significant effects.

Purpose To characterize the orientation dependence of magnetization transfer (MT) measures in white matter (WM) and propose a first correction method for such measures. Methods A characterization method was developed using the fiber orientation obtained from diffusion MRI (dMRI) with diffusion tensor imaging (DTI) and constrained spherical deconvolution (CSD). This allowed for characterization of the orientation dependence of measures in all of WM, regardless of the number of fiber orientation in a voxel. Furthermore, a first correction method was proposed from the results of characterization, aiming at removing said orientation dependence. Both methods were tested on a 20-subject dataset and effects on tractometry results were also evaluated. Results Previous results for single-fiber voxels were reproduced and a novel characterization was produced in voxels of crossing fibers, which seems to follow trends consistent with single-fiber results. Unwanted effects of the orientation dependence on MT measures were highlighted, for which the correction method was able to produce improved results. Conclusion Encouraging results of corrected MT measures showed the importance of such correction, opening the door for future research on the topic.

Soil contamination with heavy metals is becoming an increasingly global issue, as elevated levels of heavy metals exceeding regulatory limits are diminishing agricultural productivity and consequently endangering human health and the environment. This study aimed to assess the seasonal dynamics, accumulation, and translocation of heavy metals (As, Cd, Cr, Cu, Fe, Pb, Ni, and Zn) in the organs of white clover (Trifolium repens L.) depending on soil content. The heavy metal content in soil and white clover samples was determined using atomic absorption spectrophotometry. The results revealed that the analyzed soil was significantly contaminated with Cu and Zn in the industrial zone (GS) and with Pb near the urban landfill (D3). The content of heavy metals in white clover varied depending on the location and season. It is evident that white clover accumulated certain metals in its roots and leaves during spring, summer, and autumn, including Pb, Cr, Cu, Fe, Ni, and Zn. The analysis results indicated that Fe was the predominant element in white clover roots, while Zn and Fe were most abundant in the leaves. Values of BCF >1 and TF >1 for Zn suggest that white clover is a potential accumulator of this heavy metal.

Systemic lupus erythematosus (SLE) is a disease associated with an impaired autoimmune response; the immune system attacks erroneously own tissues, which leads to inflammation, tissue damage and complement activation. The latter plays a pivotal role in SLE pathology, as complement level is suited as histological marker for disease diagnoses and management. Besides, environmentally factors have been highlighted and their significant contribution for individual genetic predisposition has been pointed out. Here complement factors, their activity and their ability to modify DNA with histone proteins are reviewed; known gene mutations involved in SLE, and new therapeutic approaches suggested for SLE are discussed and summarized, as well.

Peripartum depression (PPD) has been the focus of many studies, clinical interventions and policymaking over the past decades, yet more work is needed in order to mitigate this disorder and its negative consequences. This symposium offers an examination, through various review techniques, of five emerging understudied topics related to PPD diagnosis, treatment and policy implications. Through a scoping review, the first presentation considers how economic evaluation can inform the development of mental health policy strategies and guidelines and identify implications for preventive, screening and treatment strategies for PPD. The second presentation increases understanding of PPD disorder diagnosis through a synthesis of current literature on issues regarding PPD diagnosis in different aspects. The third presentation focuses on Paternal PPD (P-PPD) and presents a consensus report on emerging issues and questions on the prevention, diagnosis and treatment of P-PPD The fourth presentation presents, through an umbrella review of systematic reviews and meta-analyses, an overarching synthesis of the literature regarding complementary and alternative therapies (CATs) to treat PPD, while assessing the extent to which CATs are associated with depressive symptoms reduction during pregnancy or after childbirth. The fifth presentation focuses, through a systematic review, on the effectiveness of Third Wave cognitive behavioral psychological interventions in treating peripartum depression. All five presenters will discuss the major findings of their work as well as potential implications for research, clinical practice and policy. Collectively, this set of papers advances the understanding of PPD diagnosis, prevention and treatment and of implications for clinical practice, research and policy.

An atrioventricular valve insufficiency is a common form of cardiac disorder in mammals, whereas it has rarely been reported in reptiles. A two-year-old male bearded dragon (Pogona vitticeps) was presented with acute onset of lethargy, inappetence and general weakness, increased respiratory effort, dark skin color, and arched back. A gastrointestinal foreign body was suspected, but radiography showed coelomic fluid accumulation. Ultrasound heart examination revealed turbulent blood flow at the level of the atrioventricular valves. Electrocardiography by modified Einthoven lead system (lead configuration 1) as described for Squamata, did not show significant changes. A tentative diagnosis of congestive heart failure as a result of valvulopathy was made. Treatment consisted of parenteral furosemide and fluid administration and assisted feeding with appropriate food intended for adult bearded dragons. The patient was weekly evaluated and successfully recovered over the four months. Echocardiography and electrocardiography were repeated 12 months later. The prescribed treatment was successful with complete resolution of clinical signs, and long-term surveillance of the bearded dragon (Pogona vitticeps).

Background: Despite the overall success of THA, between 5 and 20% report unsatisfactory results. Several factors may cause this variable outcome. 1 of them might be ethnicity which, because of its potential social impact on living conditions, may influence quality of life too. It should be studied whether patients born and being operated in their home country Bosnia and Herzegovina (BH) had similar results as immigrants being operated in Sweden (IS). Methods: Data of 280 patients were collected prospectively from questionnaires in the BH group. Patients of the IS group were eligible if both of their parents were born outside the Nordic countries, not having Swedish as their native language. Data were gained from the Swedish Arthroplasty Registry (SAR), 449 patients were included. Outcomes were pain VAS, satisfaction VAS, EQ-VAS, and the EQ-5D. Logistic and linear regression models including age, sex, diagnosis, type of fixation, surgical incision, marital status and educational level were analysed to compare those 2 groups. Results: There were considerable differences in patient demographics between the 2 groups. Before the operation, patients in the BH group reported more problems with self-care and usual activities, even after adjustment for confounding factors (p < 0.0005). Patients in the IS group reported a higher EQ-VAS and more pain VAS (p < 0.0005), the difference in the EQ-VAS was not significant after adjustment for confounding factors (p = 0.41). After 1 year patients in the BH group reported better scores in all dimensions of the EQ-5D (p ⩽ 0.005) apart from self-care. After adjustment for confounding factors, patients in the BH group were more satisfied too (p < 0.0005). Conclusions: Immigrated patients (IS group) seemed to experience less benefit from THA 1 year after the operation despite more symptoms preoperatively. There were considerable limitations affecting the results. Nevertheless, the data are a point of concern, and it is suggested to take more multidimensional care of immigrant patients.